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Firstly, we would like to.thank youwall.tor your feedback\sent 
in response to ourdetter published in the February issue. 
The number of emails we received surprised us ina.very 
positive way. We were going to publish the summary of 
this feedback in the March issue, but it appeared we need 
more time to read carefully and analyze the number of sent 
letters. 


The March issue is a compilation of articles on various 
topics. We hope that because of this variety, each of you 
will find in this issue something interesting. The title of the 
issue was inspired by Rob Somerville’s article, which is the 
last part of his.series dedicated to security foradmins. If you 
enjoyed this series oryou have any comments, please send it 
to us or Rob. 


In What's New Juraj Sipos described us the newest 
release of his project MaheshaBSD. If you are not familiar 
with MaheshaBSD yet, | recommend you to download it free 
from author's website and have some fun. 


In Developers Corner this time you won't see any well 
known name of ours regular contributors, but you will find 
there a brief overview of GhostBSD. Again, if you haven't try 
it yet - maybe you will do it after reading this short article. 


In BSD Certification series Dru Lavigne will discuss how 
to prepare for the BSDA certification exam. | hope it will be 
a helpful piece of knowledge for those who are considering 
taking this exam. 


The rest of issue is filled with articles presenting practical 
knowledge. How To section will give you the opportunity to 
try out the described techniques and solutions. From Carlos 
Neira article you will find out what to do when you need 
to debug the program and you don't have the source code 
for it. Toby Richards will take you into the journey with HPC 
cluster called Beowulf. Luca Ferrari will show you how you 
can store your data with PostgreSQL. 


From Giovanni Bechis’ article in Tips & Tricks section 
you will find out how to configure OpenBSD and NPPPD to 
provide PPTP and L2TP VPN’s in a few easy steps. This piece 
collected a very good reviews, so you can’t miss it! 


We wish you enjoy the reading and have some fun with 
your BSD after it! 


Patrycja Przybytowicz 
& BSD Team 
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What’s New 


O06 MaheshaBSD-2.0 - What’s New On The 

Lake Manasarovar? 

By Juraj Sipos 
To readers who have not yet come across the 2010 
May issue of the BSD Mag, where MaheshaBSD-1.0 
was first introduced, | reiterate that MaheshaBSD is a 
free homemade project — a Live CD based on FreeBSD 
that puts together the Hindu feel and FreeBSD. A few 
things give it this touch — for example, a possibility to 
use 4 keyboard layouts also with Devanagari (an Indian 
script used for writing Sanskrit and contemporary Indian 
languages) and IAST (transliteration of Sanskrit), the 
author's Xmodmap solution. Its name is derived from 
Mahesha, one of the names of Lord Shiva. 


Developers Corner 


12 GhostBSD: A Brief Overview 
By Nahuel Sanchez 

GhostBSD was created to encourage the use of FreeBSD 
users with little experience, and also for those curious 
who want to learn freebsd in a simple, or for those seeking 
a more robust alternative to the current options available 
in Linux kernels. An operating system with graphical 
environment, simple and useful, as is implemented in 
GhostBSD, it helps enthusiasts to take their first steps, 
provides more security and incentive to experiment. 


BSD Certification 


14 How Do| Study for the BSDA 
Certification? 
By Dru Lavigne 
The previous article in this series addressed some 
common misconceptions about certification and described 
why you should be BSDA certified. This article will discuss 
how to prepare for the BSDA certification exam. 


How To 
18 GDB(1) and Truss for Debugging 


By Carlos Antonio Neira 

Sometimes you are lucky to have the source code for the 
program you need to debug. However, there are times 
when the source code isnt available. When all hell is 
breaking loose, what do you do? On your unix machine 
there are tools that can save the day. OpenBSD, FreeBSD 
and NetBSD all have the ktrace utility for following the 
various kernel related activities of a given process. 
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202 PostgreSQL: MVCC and Vacuum 
By Luca Ferrari 

In the previous article readers have seen how to quickly 
install and configure a PostgreSQL cluster, as well as how 
to do logical backups, using pg_dump(1) and physical 
backup (with particular regard to Point In Time Recovery). 
This article shows a little more about PostgreSQL internals 
and how it exploits MVCC for high concurrency. Readers 
will also learn about the importance and usage of vacuum 
for regular maintanance. 


34 Beowulf Clusters with DragonflyBSD 
By Toby Richards 

There are two types of computing clusters: High availability 
(HA) clusters are designed so that if one computer fails, 
the other(s) take over its job. HPC clusters enable many 
computers to do the same job together so that processing 
power is increased. We're going to focus on the latter. 
An HPC cluster on consumer grade hardware is called a 
Beowulf after the classic poem written sometime between 
700 — 1000 AD. Beowulf technology is the result of a 1994 
cooperative research project between NASA and several 
universities. 


Tips & Tricks 
38 NPPPD: Easy PPTP VPN with OpenBSD 


By Giovanni Bechis 

Have you ever needed to set up a VPN for Microsoft 
Windows or Mac OS X users? From this article you will 
find out how to configure OpenBSD and npppd to provide 
PPTP and L2TP VPN’s in a few easy steps. In January 
2010, npppd was imported into the OpenBSD source tree 
and this software can act as a PPTP/L2TP VPN server 
and also as a PPPOE server. Because npppd is still under 
active development and still missing some features, it 
is not linked to the standard build yet, so to install the 
program you first need to build it from OpenBSD source 
tree. 


security 


49 Anatomy 

of a FreeBSD Compromise (Part 4) 

By Rob Somerville 
Continuing our security series, we will look at the 
vulnerabilities on our test network. From the last article, 
we discovered that to penetrate a system we continually 
needed to move from the general to the specific, and 
to identify the most vulnerable system on our network 
depending on what services were running on it 
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MaheshaBSD-2.0 


— What’s New On The Lake Manasarovar? 


To readers who have not yet come across the 2010 May 
issue of the BSD Mag, where MaheshaBSD-1.0 was first 
introduced, | reiterate that MaheshaBSD is a free homemade 
project — a Live CD based on FreeBSD that puts together the 


Hindu feel and FreeBSD. 


What you will learn... 
¢« MaheshaBSD is a modular FreeBSD rescue (Live CD) toolkit (based 
on FreeBSD 9.0-RELEASE) and it is here introduced. 


few things give it this touch — for example, a 
Rowse to use 4 keyboard layouts also with 

Devanagari (an Indian script used for writing 
Sanskrit and contemporary Indian languages) and IAST 
(transliteration of Sanskrit), the author's Xmodmap 
solution. Its name is derived from Mahesha, one of the 
names of Lord Shiva. The name Mahesha (MaheshaBSD) 
was chosen because Lord Shiva is armed with the same 
weapon as FreeBSD -— the trident. 

The Hindu feel is chosen for FreeBSD advocacy 
purposes (psychological tool) — that is, simply because 
many people who are interested in the Indian literature/ 
history/religion will find this Live CD interesting and will 
learn that, in addition to Linux and Windows, they have 
other alternatives. 


Brief Introduction Of The Project 

To quickly recap what MaheshaBSD is and how it works 
and what it offers, the following points will shine light on 
what MaheshaBSD will do for you: 

After you burn the ISO onto a CD, MaheshaBSD first 
boots into its basic MFS (Memory File System), which 
is independent of the CD/USB medium you booted 
off with. You may then eject the CD (or USB memory 
stick). You will be in a very rudimentary FreeBSD 9.0 
system running completely in memory, which is useful 
for basic system tasks (fsck, copying files, mounting 
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What you should know... 
« Some knowledge of basic commands in FreeBSD and what to do in 
case of a system crash. 


partitions, etc.). In this basic MFS environment you have 
an option to use a light version of Midnight Commander, 
mpg123 for playing mp3 files, but you may also run 
scripts and open CD/USB (file /carom/usr.uzip, which is 
in uzip compression). After running the opencd script, 
for example, the usr.uzip file gets uncompressed on the 
fly and mounted to /usr. Upon doing this, the user may 
start his/her X Window session (simply by typing startx; 
IceWM will start with the vesa video driver; however, it 
is important to say that the user must log in again from 
another console). 

MaheshaBSD is a modular (and rescue) toolkit — that 
is, it serves like a multi purpose place with several doors 








Figure 1. When you first boot MaheshaBSD off the CD, the above brief 
introduction will welcome you 
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and rooms you may go into and leave anytime. To clarify 
the concept of these doors — 1) you first boot into the 
MFS-only system; 2) you then mount the usr.uzip file 
on the CD with the open* commands; 3) you may go 
back anytime with the goback command; 4) you may 
put another CD/memory stick into your computer and 
open a different usr.uzip file on your CD/memory stick. 
For example, after running the opencd script, the user 
has an option to go back to basic MFS-only environment 
(that is — everything will be umounted including the 
usr.uzip file) and may start another open session by 
choosing from a number of available open” scripts — one 
of them (openclamcd) expands the CD with a very big 
/var directory in memory for Clamav Antivirus to work — 
this is important for its freshclam component, which will 
download these definitions from the Internet. 

After downloading the virus definitions the user may 
scan his/her computer for viruses with the clamscan 
command (clamscan -r /air) and then go back with the 
goback script to MaheshaBSD’s basic MFS environment 
and open another uzip file. 

MaheshaBSD's purpose is to bring some useful system/ 
recovery utilities to people, but on the BSD platform — like 
TestDisk (which will recover lost partitions), PhotoRec 
(which will undelete files; it can also undelete files 
on USB memory sticks), Clamav (antivirus software), 
immediate NTFS R/W access (with ntfs-3g), chntpw 
(for resetting the Windows XP/W2K passwords, a very 
practical utility), FTP server (which immediately works 
without need to configure anything), MPlayer (to watch 
films; DivX and many other codecs are supported), and 
many other things — for example, MaheshaBSD can 
be used for presentations (you can bring it anywhere 
with you and show thousands of pictures to people, or 
present videos while giving a lecture, or watch videos 
with friends), or easily let your documents speak their 
contents for you with the MaheshaBSD’s built-in speak 
(espeak) functionality. 


Simulating The System Crash 


¢ Your notebook falls down on the floor and the screen 
gets broken. You are not a techie and you do not 
know how to get your hard disk out. With the built-in 
MaheshaBSD’s FTP server (vsftpd) you may log in to 
your computer via SSH and get to your files. 

¢ You may run the Clamav antivirus software from 
within the MaheshaBSD’s environment. 

¢ You may recover lost files/partitions 
PhotoRec). 

¢ And many other possibilities... 


(TestDisk, 
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What's New in MaheshaBSD-2.0? 
MaheshaBSD-2.0 is based on FreeBSD 9.0-RELEASE, 
i386, and it was released on February 7, 2012. 

MaheshaBSD-2.0 is now Skype ready — that is, you do 
not need anything special to install to use Skype (some 
Linux libraries were missing in MaheshaBSD-1.0). You 
just download static version of Skype from the Internet 
and unpack it (download it into your /nome directory and 
then unpack it to /tmp because of memory limitations). 
Download Static Skype icon is placed on the IceWM’s 
desktop. 

Youtube videos now run without need to install Adobe 
Flash Plugin from the Internet (however, this installation is 
easy and the MaheshaBSD’s README gives instructions 
how to install it). Installation of Adobe Flash Plugin is 
recommended only in case you want to use native version of 
Adobe Flash and watch youtube videos in a better quality. 

X Window may now be started with the startxaut (start 
X automatically) script, which will generate the /etc/x11/ 
xorg.conf file (with the command Xorg -configure) and 
the X Window GUI environment will start automatically 
without any manual configuration. The problem with the 
first (after you install FreeBSD and when you generate the 
/etc/X11/xorg.conf file with Xorg -configure) configuration 
of X in FreeBSD is that users must manually write the 
following line into /etc/x11/xorg.conf (into the ServerLayout 
Section) needed for mouse to work: 


¢ Option ,AllowEmptylmput” off” 
¢ The above script (startxaut) will do this work for you. 


Some packages were removed, as MaheshaBSD-1.0 
contained more software for the same purpose (for 
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example, mp3blaster, as cmp3 offers the same functio- 
nality). MaheshaBSD-2.0 has a new logo (Manasa Devi). 
MaheshaBSD-2.0 now contains a few important Hindu 
books with icons made for them on the IceWM’s desktop 
(Markandeya Purana, Rig Veda, Devi Bhagavatam, and 
Bhagavadgita). 

MaheshaBSD-2.0 has a special Xmodmap map with 
Devanagari and IAST support; it is in the More Progs 
IceWM's menu. You may use 4 keyboard layouts with it 
(to switch between them, use CAPSLOCK). 

Seamonkey has now bookmarks for youtube videos, 
some _ Sanskrit/Hindu) resources, FreeBSD.org, and 
FreeBSD.nfo.sk. 

When you click on the Seamonkey icon, your homepage 
will be Startpage Privacy (httops:/eu3.startpage.com/) — a 
very secure search engine with Ixquick Proxy, an excellent 
privacy seal. Startpage is the European service that has 
been registered with the Dutch Data Protection Authority. 
Thus, users can access the Internet anonymously without 
need to use TOR, which is quite slow. 

When you click on the xterm icon on the IceWM's 
desktop, you will now have a larger xterm window with 
larger fonts. 

MaheshaBSD-2.0 saves more memory, aS /var and / 
etc directories are now kept in the MaheshaBSD’s basic 
MFS (/) and the opencd script does not assign any extra 
memory to these directories as in MaheshaBSD-1.0. 


Kernel is now compressed (/boot/kernel/kernel.gz). 


MaheshaBSD-2.0 has a rewritten documentation. 

A sample wpa_supplicant.conf file (to start wifi) is in the 
/etc directory. 

MaheshaBSD-2.0 has now several more useful scripts 
IN ItS /root/bin directory — for example, dos2unix (to convert 
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TXT files in DOS format to Unix format), scome (will copy 
files to and from within the MaheshaBSD’s environment 
but via SSH), burn (an example script how to burn a CD), 
findmp3 (will find all mp3 files in /mnt, will make a play list 
of them and will play them with mpg123), findogg (will do 
the same but with ogg files), html2txt (will convert HTML 
files to TXT format), swapme (will make swap in memory), 
eic. 


Brief Summary Of Most Typical Features 

Linux emulation is activated. You may run Skype or any 
Linux software under condition that you also have the 
necessary libraries. For that reason, the static version of 
Skype is recommended. 

The wired Internet should work upon startup (no wifi, 
which you must configure manually later). 

MaheshaBSD speaks. This is a very useful thing for 
hearing-impaired people, as running the command like 
espeak -f£ file.txt Will give you a possibility to hear any 
file in TXT or HTML format (to hear HTML files, put 
the -m switch immediately after the espeak command). 
| made scripts that will read the documentation (tips, 
README.html, and introduction). Just type speakintro 
(to listen to the quick introduction of MaheshaBSD), 
speakreadme (to listen to the README.html file that 
contains everything important about MaheshaBSD), or 
speaktips (to listen to some tips). 

The MaheshaBSD’s modularity feature, too, is very 
useful — you may place a tweaked mfsroot.gz file into the 
MaheshaBSD’s /boot directory (QUNZIP mfsroot.gz; mdconfig 
-a -f mfsroot md0, mount it with mount /dev/md0O /mnt, tweak 
it and gzip it back). You may then boot off your computer 
with MaheshaBSD and taste its several flavors: 1) router, 
2) FTP server, 3) web server, etc. 

The README file (it has an icon on the IceWM’s 
desktop) instructs users how to make a USB memory 
stick with MaheshaBSD. 

MaheshaBSD is not for everyday use. It is a recovery 
toolkit that can be also used for presentations, etc., and 
it serves this purpose only for a couple of hours. Its FTP 
server (vsftpd) is your door to log into any computer 
running MaheshaBSD (a broken notebook, for example) 
and save (copy) your data. You may also delete defective 
software on your Windows NTFS partition (to mount it in 
the NTFS r/w mode, use nt£s-3g — It works immediately). 

MaheshaBSD will help you be anonymous on the 
Internet (with tor and polipo [a proxy server]; just click on 
the icon of Dillo on the IceWM’s workplace and go). 

You may choose national keyboard layouts in the 
IceWM’s menu (German, Russian, Czech, Slovak); dead 
keys work too. 
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You may log into MaheshaBSD via SSH; however, only 
to your guest account. If you want to su to root account, 
you must add your guest account to the wheel group in 
yOUr /etc/group file to allow guest to su to root, or run the 
SCript /root/bin/sume that will do this work for you. 

You may write documents in the Seamonkey’s Composer 
component (HTML editor). Click on the Write documents 
icon in IlceWM. You can also download dictionaries and 
spell check your texts. 

The /boot directory, after running the open® scripts, is 
mounted via mount_nulifs and thus all kernel modules are 
available. 

Swap may be created with swapme scripts located in 
the /root/bin directory. Either type: 


freecolor 
Or 
dmesg | grep memory 


to see how much free RAM you have, then run the 
following scripts: 

Swapme (to create a 100 MB swap) 
Swapme2 (to create a 200 MB swap) 
Swapme3 (to create a 300 MB swap), etc. 

If one of them does not satisfy you, type: unswap and 
retry a different swapme script. What the swapme script 


does is: 


maconig =a =— swap: =e 100m =u.s 


swapon -a /dev/md8 


The above command will assign 100 MB to memory 
device [/dev/mas] and swapon will activate it as swap. You 
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Figure 4. MaheshaBSD running a VNC session can be also viewed ona 
Windows desktop 
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may always detach this swap with the swapoff command, 
for example: swapoff /dev/mds. 


The MaheshaBSD’s Doors 
The open” scripts were prepared by me and are in the 
/sbin directory in MFS. In addition to MaheshaBSD’s 
basic MFS, the open* scripts will assign extra memory 
tO /root ANd /home/guest directories. For this purpose, 
MaheshaBSD contains the /mts directory where all 
important directories are kept in tgz archives: /més/etc.tgz, 
jmis/etclocal:toqz, /mis/home-toz, j/mis/root.tgqz, /mis/var.tqz, 
and /mfs/varsimple.tgz. /mfs/var tgz contains the /var/db/ 
pkg (packages) database and /mfs/varsimple.tgz has its 
pkg database empty. 

The scripts (to open the MaheshaBSD’s doors) in /sbin 
are: 


¢ opencd — will mount this Live CD you booted off with 
(/dev/cao tO /cdrom) and the usr.uzip file on it (will be 
mounted to /usr). 

* opencd2 — will do the same but with the second CD- 
ROM device (/dev/cdl). 

* openclamcd — same as above, but the script will 
assign extra memory to the /var directory; this is 
needed to make room for the Clamav virus definitions 
that must be downloaded from the Internet (into /var/ 
db/clamav); the /var dir is made in memory with more 
than 100 MB for that purpose. 

¢ openclamcd2 — will do the same, but with the second 
CD-ROM device. 

¢ openclamusb — will open the USB memory stick (/dew/ 
dadsia) but with no usr.uzip mounted to /usr; you must 
have a fully populated /usr directory on your USB 
memory stick, which is particularly good for installing/ 
deinstalling packages; the /var directory can carry 
all Clamav virus definitions if downloaded from the 
Internet. 

¢ openclamusb2 — will do the same thing but with the 
second USB device (/dev/daisia). 

¢ openclamusbuzip — same as above, but with usr.uzip 
mounted to /usr. 

¢ openclamusbuzip2 — same as above but with the 
second USB device (/dev/daisia). 

¢ openda0 — a script for preparation of a USB memory 
stick in the MaheshaBSD’s environment (after you 
run it, you then just need to copy all MaheshaBSD’s 
files from /cdrom onto your USB memory stick and 
you will thus have a fully working MaheshaBSD 
on a memory stick — read the README file on 
the MaheshaBSD’s IceWM desktop for additional 
information). 
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* opendail — same as above but with the second USB 
device (/dev/daisia). 

* opendvd — same as opencd, but the script mounts 
usrdvd.uzip (it is expected that you make it yourself 
later; read the MaheshaBSD’s README. html) instead 
of usr.uzip; you will thus have, after going back to the 
MaheshaBSD’s basic MFS with the goback script, 
a possibility to mount a much bigger uzip file than 
usr.uzip on the CD. 

* opendvd2 — same as above but with the second CD- 
ROM device (/dev/cdl). 

* openmincd — this script will mount the usr.uzip file 
on the MaheshaBSD’s CD with minimal memory 
assigned to /dev/ma devices (the script assigns only 10 
MB to the /tmp directory), which is good for systems 
with low hardware resources. 

* openmincd2 — same as above but with the second 
CD-ROM device. 

¢ openusb — this will open your memory stick (/dev/ 
dadsia) with fully populated /usr dir on your stick 
(usr.uzip is not mounted to /usr). 

* openusb2 — will do the same but with the second USB 
device. 

¢ openusbuzip (or ouz) — will mount your memory stick 
(/dev/dadsia tO /usb) and usr.uzip is mounted to /usr. 

* openusbuzip2 (or ouz2) — will do the same but with 
the second USB device. 

¢ goback — will umount everything and the user returns 
to basic MaheshaBSD’s MFS as in the situation he/ 
she booted off with this Live CD (or USB memory 
stick) the first time and did not run any open” script. 


Memory Requirements 
To see memory disks attached to the system as configured 
devices in FreeBSD, type (in the console): maconfig -1. 

MaheshaBSD first goes into its basic MFS 
environment (in the root directory /). It is about 50 MB in 
size (mounted as /dev/mao to /) — a very simple (stripped) 
system without the fruits of the standard FreeBSD /usr 
contents. In this MFS — that is, before you run the opencd 
script (and other open®* scripts), you work only with 54 
MB completely in memory with a few free megabytes left 
(5.8 MB), which is not enough to download Skype and 
other goodies (like Adobe Flash Plugin). All directories 
in it are writable. 

After running the opencd script, the following directories 
will be made in memory (other scripts may bring different 
results): 


/tmp 60 MB 
/root 50 MB 
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/usr/local/etc 35 MB 
/usr/home 45 MB 
/usr/local/lib/npapi/linux-f10-flashplugin 14 MB 


RAM totally 204 MB + 54 MB (basic MFS) = 258 MB 

However, if you do not have the above memory 
available, you can always run the openmincd script, which 
creates only 10 MB for the /tmp directory — that is, 64 MB 
of RAM should suffice. 

All open* scripts (except for opendaO and opendat) 
mount /cdrom/usr.uzip (or /usb/usr.uzip) to /usr (/usr/local/ 
etc, and 
flashplugin are made writable in memory). When mounted, 
the /usr dir has the size of 1.5 GB (uncompressed), 
although the file usr.uzip (compressed) has only 583 MB. 


/usr/home/guest fjaste/local/lib/npapi/ linux-£10- 


Conclusion 

MaheshaBSD is free software but copyrighted. The 
copyright only pertains to the work made by me and not to 
packages, as licenses of these have their own conditions. 
The idea behind the MaheshaBSD project is to support 
and spread words about FreeBSD. Its Hindu touch serves 
the same purpose, because there are still many people 
who have never heard of FreeBSD. If they search for 
some Hindu keywords, they may possibly find it and try it 
and convince their neighbors that FreeBSD is not only for 
the techies. 

In the future, MaheshaBSD will always keep its original 
contours, because a possibility to type wise ideas in 
Sanskrit or IAST transliteration will make many people 
look out of their Window(s) where today, unfortunately, 
also Linux belongs. 

| thank www.rootbsd.net for allowing me to distribute 
MaheshaBSD. 


JURAJ SIPOS 

Juraj lives in Slovakia and he works in a library in an educational 
institute. Some time in the past he was fortunate to travel around 
the world and he spent a bit of time in India and Australia. Juraj’s 
hobbies are computers, mostly Unix, but spirituality too. His 
first published computer article was Xmodmap Howto (http:// 
tldp.org/HOWTO/Intkeyb/). In addition to computers, he is very 
interested in Hinduism but not really the guru side of things, but 
more-so freedom and self-actualization. More at his website: 
http://www.freebsd.nfo.sk/ (FreeBSD) 
http://www.freebsd.nfo.sk/maheshaeng.htm (MaheshaBSD) 
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My name is Nahuel Sanchez, co-founder of GhostBSD. | ly i 


gladly give you all the information you need to know about 


the GhostBSD project. 


hostBSD was created to encourage the use of 
‘e FreeBSD users with little experience, and also for 

those curious they need / want to learn freebsd in 
a simple, or for those seeking a more robust alternative 
to the current options available in Linux kernels (either for 
safety for stability or for licenses). An operating system 
with graphical environment, simple and useful, as is 
implemented in GhostBSD, it helps enthusiasts to take 
their first steps, provides more security and incentive to 
experiment, at first but then the graphical interface with 
options for system configuration finished adapting the 
code to their needs. 

The goals of the GhostBSD projects is to: 


¢ encourage the use of BSD in client’s terminals (in 
commercials) so as to augment awareness on the 
use Of Open Source software alternatives (both for 
flexibility and for cost reductions) 

¢ provide an excellent and respectable alternative to 
the field of open operating systems 


¢ promote the use of Open Source software (such as web 
browsers, word processors, email clients and so on) 
¢ spread the use of BSD on desktop computers. 















Who Are Involved in Ghost 
GhostBSD was born in the as BSD forun 
lives in Dieppe NB sana al i 
But, although we live so | 7 
in regular contact wort ng on 1 th 
ourselves by means of emails 
channel (as well as newsletter 1 
TAIL the same, the pro 
important partners have c tec biowitiileresting and 
qualified modifications. Ne\ theless, we have gone on — 
collecting opinions, sugges ions, and all types of consults 
through our web site (hi p://ghostbsd. org), for which we 
are deeply thankful to you all and be sure that we will take 
account of each of them so as to enhance the project. 
Messages via emails as well as comments left in our site, 
replies and post in our forums or conversations via the IRC 
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Figure 2. Version-1.5 
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Figure 3. Version-2.0 


channel (FreeNode##GhostBSD) are closely studied and 
assumed as valuable material for the evolution of the project. 


How GhostBSD Is Economically Solvent? 
Since it's inceptions, GhostBSD is kept in an internal 
Cannel,of distribution. Torrents and SourceForge.net were 
mainly usediuntil we had the opportunity to rent a server 
(VPS) for direct download. 

The project is still alive, thanks to three input sources: 


* One its the capital can be met in order to defray the 
costs through their own contribution, as well as through 
donations at cants via PayPal (by entering in our page) 
and also.by anybody who wish to advertise in our site. 

*« Money donations that allow us to pay hosting and 
other cost. 

¢ If you have some knowledge about programming and 
want to help us with our task, please contact us. We 
are always in a need of enthusiastic people who want 
to share their ideas and participate in the project. 


History of GhostBSD 
Version 1.0 was released in March 2010. It was based on 
FreeBSD 8 and used GNOME 2.28. 

Version 1.5 based on FreeBSD 8.1, uses GNOME 2.30. 
Compiz. The German bimonthly released magazine freexX 
(1/2011) featured GhostBSD 1.5 on a supplemented DVD 
and in an article. 


NAHUEL SANCHEZ 


Co-founder, web, External Affairs 
http://ghostbsd.org/ 
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Figure 4. Version-2.3 

Version 2.0 is based on FreeBSD 8.2, and was released 
on March 13, 2011. Some changes in version 2 include 
improvements to GDM and bug fixes. 

Version 2.5 of the final release of GhostBSD is based 
on the official FreeBSD 9.0 and is out since Jan 24, 2012. 
This version of GhostBSD has two main branches of 
the system — one is based on the GNOME desktop, the 
other on the LXDE desktop. Both are available in amd64 
and i386 versions and in form of installable CD/DVD or 
USB images. Since that month, Jan 2012, a detailed 
wiki-guide How to build GhostBSD? in combination with 
the GhostBSD toolkit is published, to build a personal 
customized version of the GhostBSD installation image, 
adding all the packages not found in the official FreeBSD 
releases, actual FreeBSD 9.0 (per january 2012). The 
GhostBSD toolkit has been designed to allow building 
of both, i386 and amd64 architectures on amd64 based 
computer systems with at least 4GB of disk space to 
swap, a sincere computing power and FreeBSD installed 
on. 

lf want a comparison tablet you can found one here: http:// 
en.wikipedia.org/wiki/Comparison_of_ BSD_operating_ 
systems. 


Short-term targets 
One of the primary objectives with GhostBSD, is to 
implement a software to install packages without ports 
(the implementation of a Network Manager). 

In other words, this provides and update software and 
new release. The project goals is to have a standard of the 
Gnome with FreeBSD and being friendly to the new user. 


ERIC TURGEON 


Founder and developer 
http://ghostbsd.org/ 
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How Do! Study for the 





BSDA Certification? 


The previous article in this series addressed some common 
misconceptions about certification and described why you 
should be BSDA certified. This article will discuss how to 
prepare for the BSDA certification exam. 


concern: there arent any training materials 

available or the training materials are too expensive. 
It explained that a psychometrically valid examination 
assesses real world skills and why the exam’s objectives 
are the ultimate study resource. 

This article describes how to prepare for the BSDA 
examination in practical terms. It describes the steps 
one can take to obtain those “real world skills” and to 
determine when one is ready to take the exam. 

When studying for the BSDA, the following steps are 
recommended: 


7 he previous article in this series discussed the 


Download the BSDA 

Certification Requirements Document 

Since the audience definition, domain percentages, and 
exam objectives are the roadmap used to create an 
examination, the document containing that information 
is your study roadmap. Finding and downloading this 
document should be your first step when studying for 
any certification exam. The document containing this 
information for the BSDA is entitled the BSDA Certification 
Requirements Document and is available for download in 
the following languages: 


¢ English: htto:/www.bsdcertification.org/downloads/pr_20 
051005_certreq_bsda_en_en.pdf 

¢ Spanish: http://www.bsdcertification.org/downloads/ 
pr_20051005_certreq_bsda_es_mx.pdf 

¢ Russian: http://www.bsdcertification.org/downloads/ 
pr_20051005_certreg_bsda_ru_ru.pdf 


This document is divided into three sections: 
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Section 1 Contains 


¢ the definition of audience for BSDA: this is a detailed 
description describing the level of experience 
required to pass the examination. When studying for 
the exam, remember that questions can not be harder 
than those that can be answered by the intended 
audience. 

¢ the operating system versions covered by the 
BSDA: this section indicates that the candidate 
needs a basic knowledge of 4 BSD operating 
systems. When setting up your study lab, you can 
install any version from the lowest number listed up 
to and including the most recent release version. 
For example, when installing FreeBSD, you can 
install any version from 4.11 (the lowest listed 
version) up to 9.0 (the highest RELEASE version as 
of this writing). 

¢ re-certification requirements: in order to meet 
accreditation requirements, a certification can not 
be for life. In other words, it must have an expiry 
date. BSDA certifications are valid for a period of 5 
years. The BSDA re-certification requirements will be 
published by Q3, 2012. 


Section 2 

Contains a description of the 7 study domains. The 
percentages for the study domains are listed at http:// 
www. bsdcertification.org/certification/associate.html. 
Table 1 lists the study domains, their percentage, and 
the number of exam objectives within each domain. 
Note that the number of objectives may not match the 
weighting as weighting indicates the importance of that 
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domain within the exam while the number of objectives 
indicates the number of testable tasks within that 
domain. 


Section 3 

Contains the objectives themselves, divided by domain. 
The next section will demonstrate how to read the exam 
objectives and use them for study purposes. 


Appendix A 

Contains an alphabetized list of all of the commands 
and files listed in the exam objectives. It also maps each 
command/file to the 4 BSD operating systems as some 
commands/files are not available in every BSD. 


Read the Exam Objectives 
The exam objectives (Section 3) begin with a section 
entitled Using the BSDA Study Domains. Read this 
section carefully as it contains detailed advice on how to 
use the exam objectives. 

Each objective has four components: 


¢ number: where the second number indicates the 
domain and the third number the objective within that 
domain. For example, 3.1.2 is the second objective 
in domain 1 (Installing & Upgrading the OS and 
Software). Domain 1 has 10 objectives and is worth 
13% of the exam. 

¢ objective: a detailed task to be assessed. As indicated 
in Using the BSDA Study Domains, watch for verbs 
(which require you to know how to do something) v.s. 
recognize (which requires you to know the name of a 
file or command). 

¢ concept: a detailed description of what the candidate 
is expected to know about that objective. 

¢ practical: the commands or files associated with 
the objective. These are also listed alphabetically in 
Appendix A. 


As an example, here are two exam objectives: 


Table 1. BSDA Study Domains 


3.2.8 Recognize BSD firewalls and rulesets. 

Concept 

Each BSD comes with at least one built-in firewall. The 
BSDA candidate should recognize which firewalls are 
available on each BSD and which commands are used to 
view each firewall’s ruleset. 


Practical 
ipfw(8), ipf(8), ipfstat(8), pf(4), pfctl(8) and firewall(7) 


3.4.1 Create, modify and remove user accounts. 
Concept 

Managing user accounts is an important aspect of system 
administration. The BSDA should be aware that the 
account management utilities differ across BSD systems 
and should be comfortable using each utility according to 
a set of requirements. 


Practical: 
vipw(8); pw(8), adduser(8), adduser.conf(5), useradd(s), 
userdel(8), rmuser(8), userinfo(8), usermod(8), and user(8) 


The first example is the eighth objective in Domain 
2 (Securing the Operating System). It begins with 
recognize, meaning that the user is not expected to know 
how to configure a BSD firewall or ruleset, but instead 
needs to be able to recognize the available tools. The 
concept clearly indicates what you need to recognize: 
which firewalls are available and which commands are 
used to view a firewalls ruleset. The practical clearly 
indicates the names of the man pages representing the 
applicable firewalls and commands. When studying this 
objective, review any man pages that you are unfamiliar 
with and compare the commands listed in the practical to 
Appendix A so that you can recognize which commands 
apply to which BSD. Since this objective only requires 
you to know how to view, don't memorize or get mired in 
the details of the (fairly lengthy) man pages as you read 
through them. Instead, focus on how to view a ruleset for 
each firewall. 


Weighting_|Number of Objectives 


1. Installing & Upgrading the OS and Software 


3. Files, Filesystems, and Disks 


5. Basic System Administration 


7. Basic Unix Skills 
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13% 10 

M% 1B 
15% 14 

a 
12% 24 

i 
17% 17 
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The second example is the first objective in Domain 
4 (Users and Account Management). It uses the verbs 
create, modify, and remove user accounts, indicating 
that the user needs to demonstrate experience in how 
to perform those three actions. The concept clearly 
indicates that the utilities vary by BSD and the practical 
lists the possible tools. This means that you should 
use Appendix A to determine which tools match which 
BSD, then practice each tool in your lab setup until you 
are comfortable using each tool to create, modify, and 
remove user accounts. 


Make a List 

As you read through the objectives, start to organize 
them in order to determine which skills need to be learned 
and how much study will be required. You may want to 
print out the document in order to write notes next to 
each objective. Alternately, you may find it easier to start 
a list that organizes the objectives into roughly three 
categories: 


° know tt 
¢ wouldn't hurt to review this 
¢ need to learn how to do this 


You should find that most of the objectives that start 
with recognize and which you don't already know, can 
go into the second category. Objectives that start with 
a verb and which vary by BSD will probably fit into the 
third category. You may wish to further separate the 
recognize objectives (which require some reading) from 
the verb objectives (which require some practice) in 
order to get a better idea of how much lab practice time 
will be involved. 

Once the objectives are categorized, you have 
your personalized study action plan. You will know 
exactly which man pages you should review and which 
commands you need to learn how to use. You can then 
decide how many objectives you can tackle at a time and 
calculate a rough estimate on how long it will take you 
to become comfortable with the material covered by the 
exam objectives. 

lt is recommended that you print out Appendix A and 
mark the commands that you need to review or learn how 
to use. Once you have worked your way through those 
commands, you are more than ready to take and pass the 
BSDA certification exam! 


Setup Your Study Lab 


When studying, you will be reading man pages, comparing 
their contents to what is required by a specific exam 
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objective, and practicing commands. You do not need 
access to a BSD system in order to read man pages as 
each BSD provides online man pages: 


¢ FreeBSD: http:/www.freebsd.org/cgi/man.cgi 

¢ NetBSD: hittp://netbsd.gw.com/cgi-bin/man-cgi 

¢ OpenBSD: http://www.openbsd.org/cgi-bin/man.cgi 

¢ DragonFly BSD: http://leaf.dragonflybsd.org/cgi/web- 
man 


Online man pages provide a convenient way to compare 
the same man page for each BSD simultaneously using 
a tabbed web browser. The online versions also contain 
hyperlinks to other man pages mentioned in the SEE 
ALSO section, making it easy to quickly learn more 
about a topic that interests you. 

In order to practice commands, you will need access 
to each BSD operating system. Each BSD can be 
downloaded for free from that project’s website. You do 
not need multiple machines in your study lab, as each BSD 
can be installed as a guest within a virtual environment. 
Possible virtual environments include: 


VMWare 

Free, commercial product. Downloads for Windows 
and Linux are available from http:/Wwww.vmware.com/ 
products/player/. 


Virtualbox 

Free, open source application. Downloads for Windows, 
Mac OS X, Linux, and Solaris are available from https:// 
www. virtualbox.org/wiki/Downloads. BSD versions are 
available as ports, packages, and PBls. Easy to use, but 
requires a good amount of RAM if you will be running 
multiple BSD guests at the same time. 


qemu 

Free, open source application. Command line by default, 
but GUI versions (aqemu, kqgemu, and qemu-launcher) 
are also available. Allows you to run multiple BSD guests 
with minimal RAM requirements. BSD versions are 
available as ports, packages, and PBIs. 

When setting up your virtual environment, you will need 
to configure the network interface as a bridged adpater 
in order to access the network using the guest operating 
system. 

To assist you in quickly creating a study lab, the BSD 
Certification Group offers a BSDA Study DVD. This DVD 
is updated every 6 months or so to the latest RELEASE 
version of each operating system. The current version of 
the DVD contains the following: 
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¢ FreeBSD 8.2, including ports collection 

¢ NetBSD 5.1, including pkgsrc 

¢ OpenBSD 5.0, including packages 

¢ DragonFly BSD 2.10.1 including pkgsrc 

¢ BSDA Exam Objectives (pdf) 

¢ BSDA Command Reference (pdf) 

¢ Psychometrics Explained (pdf) 

¢ BSDA Task Analysis Survey Report (pdf) 

¢ BSD Usage Survey Report (pdf) 

¢ BSDA Test Delivery Survey Report (pdf) 

¢ BSDP Job Task Analysis Survey Report (pdf) 

¢ BSDP Certification Requirements (pdf) 

¢ FreeBSD Handbook (pdf) 

¢ FreeBSD FAQs (pdf) 

¢ The Complete FreeBSD (pdf) 

¢ NetBSD Guide (pdf) 

¢ DragonFly BSD Guide (pdf) 

¢ pkgsrc Guide (pdf) 

¢ OpenBSD FAQ (pdf) 

¢ Latest draft of the wiki version of the BSDA Study 
Guide (pdf) 

¢ Detailed instructions on how to setup the lab 
environment and networking using qemu/aqemu 


It should be noted that each of the items on the 
DVD is freely available from the BSD project and 
BSD certification websites. The DVD is meant to 
be a convenience as well as a way to support BSD 
certification as all proceeds are used to pay for the 
ongoing psychometric maintenance of the exam. DVDs 
can be purchased for $40 USD + shipping from http:/ 
www.bsdcertification.org/store/ 


Get Your Questions Answered 

Once you have prepared your study action plan and 
configured your lab setup, you need to find the time to 
review and learn the objectives until you understand them 
and can accomplish the required tasks. Most of this learning 
can be achieved with practice, but occasionally you will 
come across something that you are not sure about. 

Like most open source projects, the BSD Certification 
project is comprised of a large community of volunteers 
who share a common interest (in this case, system 
administration of BSD operating systems). Several 
resources are available if you have a question regarding 
the understanding of an exam objective: 

e IRC: the #pbsdcert Channel is available on IRC 
Freenode. 
¢ Linkedin: a LinkedIn group of working professionals 


who are interested in BSD certification is available 


www.bsdmag.org 





at http://www.linkedin.com/groups/BSD-Certification- 
1600767 Once you become BSDA certified, 
you can also join the LinkedIn group for BSDA 
certified professionals  (htto://www.linkedin.com/ 
groups?gid=1600807). 

¢ Facebook: if you use Facebook, you can join the BSD 
certification community at httos:/www.facebook.com/ 
groups/55432547309/ Exam events are also listed 
here as they are arranged. 

¢ study wiki: a wiki where volunteers contribute 
tips for each objective is available at http:// 
bsdwiki.reedmedia.net/wiki/Table_of_Contents.html. 
lf you would like to contribute to the wiki, you can 
request the registration password on the bsdcert IRC 
channel or Facebook group. 


Even if you don’t encounter any questions while studying, 
you are welcome to join the BSD certification community 
using any of these resources. 


Summary 

This article provided practical tips for preparing for the 
BSDA examination. Once you have finished reviewing 
and practicing the exam objectives, you are ready to take 
the exam. 

The next article in this series will describe where to 
take the exam and how to arrange for an exam if there 
currently isn't an examination event or testing center near 
your location. 


DRU LAVIGNE 

Dru Lavigne is author of BSD Hacks, The Best of FreeBSD 
Basics, and The Definitive Guide to PC-BSD. As Director of 
Community Development for the PC-BSD Project, she leads the 
documentation team, assists new users, helps to find and fix 
bugs, and reaches out to the community to discover their needs. 
She is the former Managing Editor of the Open Source Business 
Resource, a free monthly publication covering open source and 
the commercialization of open source assets. She is founder and 
current Chair of the BSD Certification Group Inc., a non-profit 
organization with a mission to create the standard for certifying 
BSD system administrators, and serves on the Board of the 
FreeBSD Foundation. 
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GDB(1) and Truss for 
Debugging 


Sometimes you are lucky to have the source code for the 
program you need to debug. However, there are times when 


the source code isn't available. 


What you will learn... 

« Atechnique for debugging programs without source code 
« How to see system calls invoked by a process 

« Some basic gdb(1) commands 


hen all hell is breaking loose, what do you do? 
VV On your unix machine there are tools that can 

save the day. OpenBSD, FreeBSD and NetBSD 
all have the ktrace utility for following the various kernel 
related activities of a given process. FreeBSD has a tool 
specifically for tracing system calls. It’s called truss(1) and 
when used together with gdb(1) it can give you a clearer 
view into a black box. 

This is not specifically a truss(1) tutorial; you can check 
the man page for truss(1) for more details; here we are 
just scratching the surface (Figure 1). 

Let me give you an idea of what truss(1) can do. As the 
man page says, truss(1) traces all the system calls invoked 
by the specified process we want to look at. Let’s see — | 
have moused daemon in my unix box, let’s check it out. 

First we need to obtain the PID for the moused daemon 
and then just type: 


ster 


Se Ps 

truss [-facdDS) [-o 3 8 1 -p Wei 

truss [-faedDS! [-o 2 eo ee 0! fargsh 
is. il i tem ca | i. oa | 


DESCRIPTION 
Ihe trus 


iss utitity ft ; the 


Figure 1. Truss manpage extract 
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What you should know... 


« Some assembly language for x86 


truss -p <pid of the process you want to take a look> 


We are looking right now at the syscalls and their 
arguments (Figure 2). 

You want to know the return value of the syscall? or 
check if something is wrong? You can use gdb(1) for that! 
You don’t have the source code? No problem, you can 
look at the registers. The return value of most system calls 
and program functions is stored in the seax register (I am 
referring to x86 architecture). 

| have written a small program that we will use as an 
example. It simply outputs the sum of the variables in 
a for() loop — pretty simple but enough for this proof of 
concept. Here is the code: Listing 1. 

Save this code to a file. | called it test.c (very original). 
If you have installed make(1) and a C compiler, you just 
need to type: 
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# make test 
or as usual, do a 
# cc test.c -o test 
(notice that | have omitted the -g flag so we don't have 
any debug information generated). 

Now that you have compiled the source let's inspect this 
with gdb(1). 
# gdb test 
We set our first break point at the main function. As you 
recall, we don’t have the source code for this, so we are 
blindfolded and looking for any clue that might help us 
resolve a problem. 


(gdb) b main 


We type r (run) and start the program flow... 





Listing1. Test.c example source code 


ZiMelude<SEOlo. lM 


Pie  OUmInY, (Ol, OA, 03) 
{ 
pe weno 
tmp=p1+p2+p3; 
printf ("dummy value: %d\n",tmp) ; 


return tmp ; 


Dae Matn () 
{ 
Ime pl=l)p2—-2, ps—5, 1, emp; 


for (i=0; 1<=10; i++) 
{ 
tmp=dummy (p1,p2,p3); 
printf ("dummy() returned d\n", tmp) ; 
Dl +; 
Oa tey 
pot 
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Listing 2. Using gdb disas command for dumping of assembler 
code for function main 


(gdb) 


disas 


Dump of assembler code for function main: 


0x080483d0 
0x080483d4 
0x080483d7 
0x080483da 
0x080483db 
0x080483dd 
0x080483de 
0x080483el 
0x080483e8 
0x080483ef 
0x080483f6 
0x080483fd 
Ox080483ff 
0x08048402 
0x08048406 
0x08048409 
0x0804840d 
0x08048410 
0x08048413 


0x0804841b 
Ox0804841le 
0x08048422 


0x08048429 


0x0804842e 
0x08048432 
0x08048436 
0x0804843a 
0x0804843e 
0x08048442 
0x08048444 
0x08048447 
0x08048448 
0x08048449 
0x0804844c 


End of assembler dump. 


<main+0O>: 
<maint4>: 
“Madina. 
iievieger | eee 
“Melba lee 
<Malnt | 323 
<maintl4>; 
<Matnnh = 
<main+24>; 
Menta lee 
<maint+38>: 
<maint4 >>; 
<maint+4 />; 
<maint+50>: 
<main+54>; 
Meio 1s 
Mehl aoe. 


<maint+64>: 





<maint+67/>: 


<Matn+ 15> 


<maint+7/8>: 


<maint+82>: 


<maint89>: 


<maint+94>: 


<maint98>: 


AmMeatns W02>: 
<main+1l06>: 
AMa tnd POs 
<maint+1l14>: 
“Meme Lies: 
<Malnr LoS: 
Mea tide 


<maintl21>: 








<maintl24>: 


lea 
and 
bush! 
push 
mov 
push 
sub 
movl 
movl 
movl 
movl 
jmp 
mov 
mov 
mov 
mov 
mov 
mov 


call 


Mov 
MOV 


movl 


call 


addl 
addl 
addl 
addl 
cmpl 
jle 
add 
pop 
pop 
lea 


ret 


0x4 (SesSp) , 6ECX 
SOxfffffff0, tesp 
-0Qx4 (%ecx) 

Sebp 

sesp, sebp 

SECX 

$0x34,%esp 
SO0x1,-0x18 (%ebp) 
S0x2,-0x14 (%ebp) 
$0x3,-0x10 (%ebp) 
$0x0,-Oxc (%ebp) 
0x804843e <main+110> 
-0x10 (sebp) , seax 
Seax, 0x8 (Sesp) 
-0x14 (Sebp) , seax 
%eax, 0x4 (Sesp) 
-0x18 (Sebp) , seax 
eax, (SESP) 


0x80483a4 <dummy> 


-0x8 (sebp) , seax 
%eax, 0x4 (Sesp) 


$0x8048510, (%esp) 


0x80482d8 <printf@plt> 


SOx1,-0x18 (%ebp) 
SO0x1,-0x14 (%ebp) 
S0x1,-0x10 (%ebp) 
S0x1,-Oxc (%ebp) 
S$0xa, -Oxc (%ebp) 
Ox80483ff <main+47> 
50x34, %esp 

SECX 

sebp 


-0x4 (%ecx) , Sesp 
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(gdb) r 


Breakpoint 1, 0x080483de in main () 
Current language: auto; currently asm 


#0 Ox080483de in main () 


So we hit our break point. We can only can see the asm 
instructions in this program, so we type: Listing 2. 

The hexadecimal values in the column on the far left are 
the addresses of instructions which will be executed as 
the program runs. These values will probably differ from 
what you'll see if you’re running this code. We can use 
these addresses for setting breakpoints. 

In the output we see a function called aummy ). Let’s put 
a break point there (Listing 3). 

So here we see that this function calls the classic prints () 
function and puts the return code in the seax register. 

To see the contents of all registers just type: Listing 4. 

We step through the aummy() function until we pass the 
call to prints (). We can then inspect the return value by 


typing: 


(gdb) p Seax 





Listing 3. Setting a breakpoint at the dummy function 


(gdb) b dummy 
Breakpoint 2 at 0x80483aa 


Let's check the asm for the dummy function. 
(gdb) disas dummy 

Dump of assembler code for function dummy: 
0x080483a4 <dummy+0>: 
0x080483a5 <dummyt+1>: mov 
0x080483a7 <dummyt+3>: sub 


push Sebp 
Sesp, sebp 


$0x18,%esp 


0x080483aa <dummy+6>: mov Oxc(%ebp) , sedx 
0x080483ad <dummy+t9>: mov 0x8 (%ebp) , seax 
0x080483b0 <dummyt+12>: add ‘%edx, %eax 

0x080483b2 <dummy+14>: add 0x10(%ebp) , seax 


0x080483b5 <dummy+17>: mov 
0x080483b8 <dummy+20>: mov 
Ox080483bb <dummy+23>: mov 
Ox080483bf <dummyt+27>: movl 
0x080483c6 <dummyt+34>: call 
0Ox080483cb <dummy+39>: mov 
Ox080483ce <dummyt+42>: 
Ox080483cf <dummyt+43>: ret 


6eax, -0x4 (%ebp) 
-0x4 (%ebp) , seax 
Seax, 0x4 (Sesp) 
$0x8048510, (%esp) 
0x80482d8 <printf@plt> 
-0x4 (%ebp) , seax 


leave 
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To modify the integer value passed to the print£() 
function we set a break point at the instruction that 
pushes the scax register onto the stack and then change 
the value in the register. To do this, we need to use the 
hexadecimal address when setting the breakpoint. This 
is the instruction where we want execution to stop: 
0x080483bb <dummyt+23>: mov %eax,0x4(%Sesp) 

so we set the break point using the hexadecimal address 
of the instruction: 


(gdb) break *0x080483bb 


Now we can modify the seax register value to 99 and let 
the program continue running. 


(gdb) set Seax=99 
(gdb) c 


dummy value: 99 
(additional gdb output ignored) 


dummy () returned 6 





Listing 4. Dumping the registers content 
(eel) 


Breakpoint 2, 0x080483aa in dummy 


eax Ox2 Z 

eCcxX 0x0 0 

edx Oxb7f£8f£0£0 = A042 7160 
ebx Oxb7f£8dtf4 -1208426508 
esp Oxbft9bb410 Oxbft9bb410 
esi 0x8048460 L345 13760 
edi 0x80482f0 BOA Se 37 
eip 0x80483aa 0x80483aa <dummy+6> 
eflags OxXZe2 be lea 

es xis lige 

SS Osis ZS 

ds Ox7b 123 

es OpiS LAs 

fs 0x0 0 

gs Oxa0 BL 
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The first line of output shows that prints() used the value 
in the seax register (99). The value of the variable tmp did 
not change, as can be seen by the output when print£() 
is called from the main() function (tmp = « in this particular 
iteration). 

To modify the return value of the aummy() function in the 
program, we need to change the value of the scax register 
just before exiting the function. This is done by setting a 
breakpoint at the /eave instruction. 


Ox080483cb <dummy+39>: mov  -0x4(%ebp), teax 
0Ox080483ce <dummyt+42>: leave <-- here !! 
Ox080483cf <dummy+43>: ret 


As before, we need to set the breakpoint using the 
address of the instruction. 


(gdb) break *0x080483ce 
(gdb) set Seax=999 


Now, when the program continues,the return value of 
dummy() iS Output by print£() aS: 


dummy () returned 999 


You can modify the flow of the program using the 
registers as the program checks for a file; or in the case 
of a routine that returns an error code, you can override 
the result easily. 

So it is basically wash, rinse and repeat until you find the 
bug or the condition that triggers the failure and is giving 
you a headache. This is just the tip of the iceberg, but it 
is pretty helpful playing with the registers and truss if you 
don't own the application that is causing the problems. 

| hope this was helpful. It’s brief but the documentation 
is always the best source for information. Just calling 
up the ol’ man page can show you new possibilities to 
explore apart from these. 
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The BSD Certification Group Inc. 
(BSDCG) is a non-profit organization 
committed to creating and 
maintaining a global certification 
standard for system administration 
on BSD based operating systems. 


@ WHAT CERTIFICATIONS ARE AVAILABLE? 


BSDA: Entry-level certification suited for candidates 
with a general Unix background and at least six months of 
experience with BSD systems. 


BSDP: Advanced certification for senior system administrators 
with at least three years of experience on BSD systems. 
Successful BSDP candidates are able to demonstrate 

strong to expert skills in BSD Unix system administration. 





@ WHERE CAN! GET CERTIFIED? 


We’re pleased to announce that after 7 months of 
negotiations and the work required to make the exam 
available in a computer based format, that the BSDA 
exam is now available at several hundred testing centers 
around the world. Paper based BSDA exams cost $75 USD. 
Computer based BSDA exams cost $150 USD. The price of 
the BSDP exams are yet to be determined. 


Payments are made through our registration website: 
https://register.bsdcertification.org//register/payment 


GD WHERE CAN | GET MORE INFORMATION? 


More information and links to our mailing lists, LinkedIn 
groups, and Facebook group are available at our website: 
http://www.bsdcertification.org 


Registration for upcoming exam events is available at our 
registration website: 
https://register.bsdcertification.org//register/get-a-bsdcg-id 
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MVCC and Vacuum 


PostgreSQL: 


In the previous article readers have seen how to quickly 
install and configure a PostgreSQL cluster, as well as how to 
do logical backups, using pg_dump(1) and physical backup 
(with particular regard to Point In Time Recovery). 


What you will learn... 
¢ what MVCC is and how it is exploited in PostgreSQL 
¢ how to deal with Vacuum and Auto-Vacuum 


internals and how it exploits MVCC for high 

concurrency. Readers will also learn about the 
importance and usage of vacuum for regular maintanance. 
The database used in the examples can be rebuilt at any 
time using the simple script in Box 1. 


7 his article shows a little more about PostgreSQL 


What you should know... 

« basic SQL concepts 

« how to configure and access a PostgreSQL instance 
¢ basic shell commands 


MVCC 

MVCC stands for Multi-Version Concurrency Control and is a 
technique that PostgreSQL uses to provide high concurrency 
while keeping database consistency. Giving a full explanation 
of how MVCC works is out the scope of this article, please see 
the official manual and the references for further readings. 





BEGIN; 
CREATE TABLE IF NOT EXISTS magazine (pk serial NOT NULL, 
id text, 

IO ase eet ee 

issuedon date, 

Pipe srext, 

PRIMARY KEY (pk), 

UNIQUE (id) 

); 

TRUNCATE TABLE magazine; 

INSERT INTO magazine (pk, id, month, issuedon, title) 
WAH Sete eo Ole les ile, pe Opa On ie ee alaliacam, 
INSERT INTO magazine (pk, id, month, issuedon, title) 
VASO Sap Oe ale Pe OnE = On Nea claecas, 
INSERT INTO magazine 


(Chal, MOnchi lS sSuedom, srl le) 
AUIS (35 aoe Ole ee ee 
COMM aT: 


2 OZ Se cae, 
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Box 1. Content of the magazine.sq] text file used to reload the data (file magazine.sq]). 


‘FreeBSD: Get Up To Date’); 
‘Rolling Your Own Kernel’); 


‘Speed Daemons’); 
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Databases must ensure that even when clients are 
executing concurrent statements on the same set of 
data, the latter remains consistent. This usually requires 
locking: briefly the first client that gets access to the 
data locks it so that other clients have to wait until the 
lock is released to get access to the same data. Locking 
can quickly become a bottleneck and can lower the 
concurrency of the queries. MVCC takes another 
approach to the problem: each client has access to a 
private snapshot of the data. Trhough snapshots multiple 
versions of the same data (e.g., the same tuples) are 
available to concurrent clients. This does not eliminate 
locks at all, but dramatically reduces the need for them 
in the backend. Simplifying MVCC can be compared to 
Copy-On-Write (COW) filesystems like ZFS: each time a 
tuple is going to be manipulated a clone is created and 
changes are applied to the latter. 

PostgreSQL numbers each transaction with a 32 bit 
progressive identifier called xid; moreover within each 
transaction each statement is also progressively identified 
(cid). It is worth reminding that each statement is always 
executed in a transaction context, either explic (i.e., 
issuing a secin) Or implic (no szcin has been issued). 

PostgreSQL keeps track of MVCC attaching to every 
tuple metadata fields: xmin, xmax, cmin and cmax. Such 
fields are available to the user but are hidden in each 
seLect Statement until not explicitly named (see Listing 1). 
The meaning of each metadata field is the following: 


* xmin indicates the transaction identifier that created 
the tuple; 


* xmax indicates the transaction identifier that 
invalidated or is going to invalidate the tuple (via 
UPDATE OF DELETE); 

* cmin indicates the command 
transaction that created the tuple; 

* cmax indicates the command identifier within a 
transaction that invalidated the tuple (i.e., either 
updated or deleted the tuple), if the tuple has been 
invalidated. 


identifier within a 


To get an idea of how metadata is stored, consider 
the query shown in Listing 1: you can see that all the 
tuples have been created by the same transaction 727, 
with three consecutive inserts (Cmin and cmax range 
from O — first command — to 2 — last command within 
the transaction) and has not been yet invalidated (i.e., 
xmax is still 0). You can also see that the transaction 
727 was executed 5 transactions before the current 
one: the age) function returns the distance (in terms 
of transactions) from the current transaction to another 
transaction identifier and the function ¢xia— current () 
returns the identifier for the current transaction 
(therefore the sztecr statement is executed as implicit 
transaction 732). In other words, five transactions ago 
there was an explicit transaction numbered 727 that 
loaded the three shown tuples (with three consecutive 
statements) and nothing more changed such tuples. 
Now consider doing an implicit transaction that updates 
the tuple with pk = 17, as reported in the second half 
of Listing 1. What happens is that the tuple with xmin 
727 was substituted by the new copy of the same tuple 





Listing 1. Evaluating MVCC data for each tuple. 


bsdmagdb=# SELECT xmin, cmin, xmax, cmax, age(xmin), 
bsdmagdb=# SELECT xmin, cmin, xmax, Cmax, age(xmin), 


CME Renan xMax| sella x | age a ixucdicumrent al ok 


------ $------ f------ $------ $----- $-------------- +---- 
Tan | 0 | Onn Oo Dal Vos eat 
Hela) ie, On| 1 | Daal hoe Ne 2 
daa | Zell 0 | ae i) US7 |ees 


bsdmagdb=# UPDATE magazine SET title = 'FreeBSD: Get 
bsdmagdb=# SELECT xmin, cmin, xmax, cmax, age(xmin), 


Sin) | enim ||) xtiax || emax | age ||) exidecurrent |) pk 


------ $------ $------ $------ $----- $-------------- $---- 
TZN de Or] | aa | 134 Wh 2 
Tod | oil O77 Zon | 7c es 
TOSI Om 0 | 0 | | 7a WP al 





EX iG (Cu prent() > sehOM maga ime; 


EXid eumrent (7 ROM maga 7 ae livin 337 


| id | title 

+--------- $------------------------- 
[20d = Oie i 1 | FreeBSD: Get Up To Date 
202: 12 | Rolling Your Own Kernel 
Oa an) 11 | Speed Daemons 


Up To Date!' WHERE pk = 1; 


PSG (Cumrent (ela, welel eu nROM magazine - 


| sh | title 

+--------- $-------------------------- 

| 2011-12 | 12 | Rolling Your Own Kernel 
p20 iy 11 | Speed Daemons 

ja 202 Oia 1 | FreeBSD: Get Up To Date! 
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(with data opportunely changed by the veparz) and now 
the tuple has xmin 733 (one transaction before the 
last szzecr in Listing 1) and both cmin and cmax are 
set to O (the first command in an implicit transaction 
is the statement itself). What happened behind the 
scenes? As shown in Figure 1 the old tuple (xmin 
727) was marked as no more valid (xmax 733) and 
a new tuple has been created (xmin 733). When you 
execute a setecr on a table PostgreSQL gives you back 
only tuples marked as still valid. To better understand 
what valid means we have to do a little more 
experimentation. To that purpose it is worth installing 
the pageinspect extension (avilable from the -contrib 
module) which allows the DBA to see how a data page 
is actually used; it is also worth installing the pgstattuple 
module to get even more information about the data 
page status. Listing 2 shows how to install the above 
modules in your own 9.1 database. As readers can see 
from Listing 2, the data page contains four tuples while 
the table in Listing 1 shows only three of them: the trick 


is that the first tuple of Listing 2 is the one inserted by 
transaction 727 and made not valid from transaction 
733 (lp = 1), which substituted it with another tuple (/p = 
4). As soon as the system realizes that tuples are only 
three (or the DBA informs the database to adjust the 
data page — more on this later) the system cleans the 
tuple in the data page to free space (see Listing 3). 
Table 1 shows the transaction isolation levels as 
defined by the SQL Standard and how PostgreSQL 
adhere to that; please note that it is fine for the 
standard to handle a isolation level with a stricter one 
and therefore before version 9.1 PostgreSQL provided 
support only for two levels, READ COMMITTED 
and SERIALIZABLE, making the other two, READ 
UNCOMMITTED and REPEATABLE READ respectively 
an alias of the formers. Since 9.1 a new level has been 
natively supported, REPEATABLE READ, that behaves 
exactly as the SERIAL/IZABLE level in previous versions. 
As readers can see snapshots are computed at different 
times depending on the transaction isolation level: for 





Listing 2. /nstalling the pageinspect and pgstattuple modules. 


~> find /usr/local/lib -name 'pageinspect.so' 
/usr/local/lib/postgresql/pageinspect.so 
yo hind 3) Uste/ Wocaly/ lib name oqstartuple.so. 
/usr/local/lib/postgresql/pgstattuple.so 
bsdmagdb=# CREATE EXTENSION pageinspect; 
bsdmagdb=# CREATE EXTENSION pgstattuple; 


PROM heap page items 
ORDER BY lp 


get vaw page Wagazane 7 0 


tp lortlags | -xmine | xia |) teeuird 


~---}---------- $------ $------ $-------- 
| lee, Oa 
Z| Lo eZ || Oi) 1 OeZ 
ery Le 2 eo Oe (0,3 
a ae osm OF On 


FROM heap page items 
ORDER BY lp 


get vaw page (magazine 7 0 


Toil elads || smiles xia |) secre id 


~---}---------- $------ $------ $-------- 
iam it |e 2 | OM eae Orel 
oa a ee OF 2 
S| il ee OR meer 





bedmagd>—; SHluCl Wp, lp wags, smi. bexe,. 10eo fo XMM, ve XMaw., Gexe. Itc Ao <Max, we ce nd 


Listing 3. /nspecting the data page after the database removed expired tuples. 


Dedmagdb—7 SHmECT pp lp tags, .6 Xmin. ,cext, inc Al XMmin; 9 XMax.,toxG., Med As x<Miax, 6 .celd 
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Table 1. Transaction isolation levels in PostgreSQL. 


Dirty Read Non-Repeatable Read Phantom Read 

An unfinished An unfinished An unfinished 

transaction canread _ transaction read transaction re- 

data manipulated by _—_ data that is then executes a query that 

another concurrent manipulated by returns a different 

transaction even if another committed set of tuples due to 

the latter has not yet — transaction, so that another concurrent 

committed. the former can no transaction that 
more read the same committed changes 
data again. that affected the 


selection criteria. 














Read NO, default 
Uncommited to READ 
COMMITTED 
Command to set isolation level SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED; 
Read Committed YES, via READ Command start. 
COMMITTED 
Command to set isolation level SET TRANSACTION ISOLATION LEVEL READ COMMITTED; 
Repeatable Read NO (before 9.1), Transaction start. 
(default) default to 
SERIALIZABLE 
YES (since 9.1), 
via REPEATABLE 
READ 
Command to set isolation level SET TRANSACTION ISOLATION LEVEL REPEATABLE READ; 
Serializable YES, via Transaction start. 
SERIALIZABLE 
Command to set isolation level SET TRANSACTION ISOLATION LEVEL SERIALIZABLE; 





initial situation 


727, cmin = 0, cmax = 0, xmax = 0 
727, cmin = 1, cmax = 1, xmax = 0 


727, cmin = 2, cmax = 2, xmax = 0 





UPDATE magazine SET title = 'FreeBSD: Get Up To Date!" WHERE pk = 1; 


xmin = 727, cmin = 6, cmax = 0, xmax = 733 


xmin = 727, cmin = 1, cmax = 1, xmax = 0 


xmin = 727, cmin = 2, cmax = 2, xmax = 6 


xmin = 733, cmin = 6, cmax = 6, xmax = 0 














Figure 1. Conceptual visualization of the changes performed in Listing 1 
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Listing 4. MVCC and concurrent transactions (READ COMMITTED) 


(in terminal A) 


bsdmagdb=# BEGIN; 
bsdmagdb=# SELECT xmin, cmin, xmax, cmax, age(xmin), 


SMEs |) crim |" xmax |) cilax |Sage |) txid icunrens || pk 


------ $------ $------ $------ $----- $-------------- +---- 
754 | Oa 0 | 0 | 3 Nee 
qo) | OF 0 | 0 | Zoi ToT Ae Z 
dao | 0 | 0 | On| lew io) io 3 

(3) rows) 

bsdmagdb=# UPDATE magazine SET title = '[SOLD OUT] ' 


bsdmagdb=# SELECT xmin, cmin, xmax, cmax, age(xmin), 


Stn | Menine | xmax \enax|—acge | txid weumpvent | ok 


------ $------ $------ $------ f----- $-------------- +---- 
754 | Oy () 0 | S| LEE Ge eel 
7 | 0 | Oe) 0 | Oi] Tes 
S| 0 | 0 | 0 | 0 Pe 2 


bsdmagdb—=; SHleClT Wp, ip tags, = mim; ; Cexe;;1me0 Ao 
PROM heap ipage theins( cer saw page| Magazine Us) 
ORDER BY lp; 

Tipe lotlags |) xmin | > xian |) fckud 


~---}---------- $------ $------ $-------- 
i) 1 | 754 | Oa eri) 
2 Le oO ie on amen ce) 
3 ee oo ae oy oa ee) 
au, ee el Ceo) 
sil eres Oper; >) 


bsdmagdb=# COMMIT; - after this B is unlocked! 
(in terminal B) 


bsdmagdb=# BEGIN; 


bsdmagdb=# SELECT xmin, cmin, xmax, cmax, age(xmin), 


AEA eilits = Max ella tal age es xi cUmreiia= || Ok 
------ +------ +------ +------ +----- +-------------- +---- 
754 | an) Om} om 4 | 15S. ey 
75 )| alee a Oma onl (33 je 2 
Toe | Oe ae Os} Jl TSN S 

(3 rows) 


bsdmagdb=# UPDATE magazine SET title = title || ' 


- the transaction is locked here until A does a COMMI 


[SOLD OUD] WHERE 1a iiike 


xe ie nie oraen() 2 Gl, uae a OM ainaigalanae, 


| ae) | title 

+--------- $------------------------- 

P2002 01) 1 | FreeBSD: Get Up To Date 
20 lees 12 | Rolling Your Own Kernel 
[= 2 One ie) 11 | Speed Daemons 


|| title WHERE id like '2011-3'; 


EMidveurreni(y) id, rite BROOM magazine, 


| ad | title 

+--------- $------------------------------------ 

20d On 1 | FreeBSD: Get Up To Date 

(ee ee 11 | [SOLD OUT] Speed Daemons 

20k? 1] 12 | [SOLD OUT] Rolling Your Own Kernel 


iin, eexMax i text ii ime AS xmax, 6 Cela 


Cig seurnen i) iC, webelew OM Magazine, 


| id | title 

+--------- $+------------------------- 
202-0 1 | FreeBSD: Get Up To Date 
[202 12 | Rolling Your Own Kernel 
Pee Oi eat 11 | Speed Daemons 


Ot =e 
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a READ COMMITTED a new snapshot is computed 
each time a command begins in order to ensure that 
the command will see all the data committed by other 
transactions; on the other hand in REPEATABLE 
READ and SERIALIZABLE mode a snapshot is created 
once when the transaction is started, so that it will 
see data committed only before the transaction itself. 
The difference between REPEATABLE READ and 
SERIALIZABLE in version 9.1 is that the former uses 
a lock on the data to avoid concurrent manipulations, 
therefore simulating a sequential execution of the 
transactions, while the latter keeps a set of so called 
predicate locks, which are locks on queries and not on 
their data. Predicate locks are used by PostgreSQL 
to understand if transactions are executing conflicting 
queries, forcing then one to abort without having to lock 
the data (and therefore providing a better concurrency). 
In order to help the system incrementing the concurrency 
it is also possible to indicate a transaction as read-only 
or write-only via the ser Transaction rsoLaTion Command. 





In order to see how MVCC works with concurrent 
transactions clean and refill the magazine table, then 
start two transactions in two different terminals (A with xid 
757 and B with xid 758); see Listing 4 for details. Imagine 
that A executes the vrparz before the one of B; since the 
default isolation level is READ COMMITTED then B has 
to wait for A to either commit or rollback, therefore the B 
uppate keeps the session locked waiting for A to conclude. 
Please note, as shown in the B terminal, that B sees the 
old version of the data (i.e., the data has not been modified 
by A permanently) but it is also informed that transaction A 
is changing the data (xmax is set to 757). In other words, 
B knows that the tuples will expire if transaction 757 (A) 
commits. Inspecting the page data readers can see how 
there are two tuples with a xmax set to 75/7 and two new 
tuples with cmin and no cmax set. 

Replaying the same experiment with a fresh situation 
and making transactions A and B serializable will report 
an error in transaction B because the vrparz cannot be 
serialized. 





Listing 5. MVCC and transaction commands 


bsdmagdb# BEGIN; 
bsdmagdb=# \i magazine.sql 


magazine; 


bsdmagdb=# UPDATE magazine SET title = '[SOLD OUT] ' || 





bedmagdb=7 SELECT xmin, cmin, xmax, Cmax, ege (xin), ~xid Current (), * FROM magazine, 


SCH ee Misenen | xMax es | meliax sl acgem | Ext cue eet spk | | title 

------ $------}------4------4-----4--------------4----4---------4------------------------- 
784 | O.7 OS] 07 Oy Oe eevee e202 Cee) | 1 | FreeBSD: Get Up To Date 
cay) HA Oe] 1 | 0 | 104 Ne 2 |) ZO? 5) 12 | Rolling Your Own Kernel 
784 | Zl Oe 7] nl 0 | (S42) So) |e 20m 11 | Speed Daemons 


bedmagdb=; DECUARE cursor mvec. CURSOR FOR SELECE xmin, cian, xmax, ciax, age (min); sexid currene () 20, tele FROM 


title WHERE id like 


bedmagdb—; Sele C! Sin, "cnin, <max,  ciax, age (iim) 7 ieXid eUrronG(), 10, siele FROMimagazine, 


Amun Pe einimys | xilax S| emax | Sages |S exidecurrent || pk. \| | title 
------ $------}------4------4-----4--------------4----4---------4------------------------------------ 
784 | 0 | 0 | 0 | On OA i le |Z Oi Oa 1 | FreeBSD: Get Up To Date 
784 | =| 0 | = a OF TSA io le 20 ie 11 | [SOLD OUT] Speed Daemons 
Yea. | 3 || Oa | 3 | Oo | Too Nine 2 |) 2012 | 12 | [SOLD OUT] Rolling Your Own Kernel 
bsdmagdb=# FETCH ALL FROM cursor mvcc; 
Kime | enin |) xmaxs|eemax | age | txid current || pk | | title 
------ $------}------4------4-----4--------------4----4---------4------------------------- 
784 | 0 | (Oe) os) 0 OA ty ele TZ Onl Os 1 | FreeBSD: Get Up To Date 
164 | Le 84 1 | 0 | TOL) O23) > ZO | 12 | Rolling Your Own Kernel 
784 | O | 434° || 0 | OF | ff OAS ees |i Ole ey 11 | Speed Daemons 


PU dao 7 
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To understand the usage of cmin and cmax we 
have to issue conflicting commands within the same 
transaction. To dO SO we USE a cursor, that is a resource 
that will fetch row by row data from a set. As Listing 
5 shows, a transaction is started, then the tuples are 
loaded and a cursor to query the table is declared. 
Since the cursor is the third command in the transaction, 
further commands will not change the snapshot seen 
by the cursor: therefore an vurpate of the tuples is 
immediately reflected in the transaction, but not in the 
cursor. In this scenario the in-transaction snapshot is 
based on the values of cmin and cmax. Of course, 
it does not make sense to compare cmin and cmax 
out of a transaction boundaries, since the transaction 
isolation level will define how snapshots are visible. As 
an implementation detail, it is worth noting that cmin 
and cmax are internally stored as a single value, so 
such information does not suffice to understand if a 
multi-statement transaction has created and expired a 
tuple. The solution PostgreSQL adopts is to keep track 
in the memory page of a so-called combo command 
id that informs that the tuple has been created and 
expired within the transaction. 


Anatomy of a Data Page 

PostgreSQL data is contained in so called data-pages 
(see Figure 2): each page has a space where tuples 
are placed that grows toward low addresses and an 
array of pointers to each tuple, called /inear pointers 


(Ip), which grows towards high addresses. When a 
specific tuple has to be found, PostgreSQL loads (if not 
already present) the data page that contains such tuple 
into a free shared buffer (a region of shared memory) 
and inspects it to find the linear pointer that leads to the 
tuple. The advantage of keeping linear pointers within 
the data page is that tuples can be re-arranged within 
the page without having to change the way to find the 
page itself. 

Listing 6 provides a simple shell script that simulates a 
workload to see how data pages change during several 
tuple operations. The workload is quite simple: starting 
from an empty magazine table, it inserts a set of tuples, 
immediately modifies them and finally deletes all of them. 
From a user perspective the magazine table is unchanged 
at the end of the workload, because it is empty. However 
the script reports the following output: 





Filenode was /postgresql/clusterl/base/16398/17110 
Size before starting: 0 
Size after insert: 262144 

[ 32 pages with 223893 bytes for 5000 live tuples] 
Size after update: 540672 

[ 66 pages with 258893 bytes for 5000 live tuples] 

(around 2 times initial size) 

Size after delete: 540672 


[ 66 pages ] (around 2 times 


initial size) 








pd_upper 
special 


pd_Linp[] 





Figure 2. PostgreSQL data page layout 
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/* Log sequence number */ 

/* begin of the free space */ 

/* end of the free space */ 
_Spe /* special space */ 

pd_pagesize version /* for compatibility */ 

/* tuple pointers */ 


special space 
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Listing 6. A script to test MVCC and how data pages changes 


#1 /oin7 sh 

PGDATA=/postgresgl/clusterl 

DBOID='oid2name -gq | grep bsdmagdb | awk '{print $1;}'' 

PAGE Po UAL expec 3 074 

FILENODE=S { PGDATA} /base/$ {DBOID}/$ { TABLEOID} 

FILLFACTOR=100 

PAGE OURR Y=) Sain! ip el patlags,) = smi text: inte AS min, ste xMex: text: 1nts Al tax, tpeiad BRO teapypage) 
Tuts (yeh e raw Page Magazine) 0) ) ORDER aie, 

echo "Cleaning the magazine table..." 

psql -U bsdmag -c "TRUNCATE TABLE magazine;" bsdmagdb 

psql -U bsdmag -c "VACUUM FULL magazine;" bsdmagdb 

psql -U bsdmag -c "ALTER TABLE magazine SET (fillfactor=SFILLFACTOR) ;" bsdmagdb 

TABLEOID='oid2name -U bsdmag -t magazine -d bsdmagdb -q | awk '{print $1;}'' 

FILENODE=S { PGDATA} /base/$ {DBOID}/${TABLEOID} 

is —ih SFILENODE 

SIZE] sole SP IRMNODE S| Pawks {print 55,7)" 

sleep 2 

echo "Inserting tuples..." 

pseqlesU bsduag =o “MSERD UNO magazine(id, title) /AnUNS\ djenesate series (1, 5000) 7 vacuum—test| 7 ss bsdmagdb 

peql-U bedmag =-psee pager—-c1n —-e¢ 75) PAGE OUERY |) ™ bedmagdb 

is 1h se ELE NODE: 

SIZE Eas) Thor I LMNODE | awk “ {print 555.) 7 

ote LOPS osgh “UW beciag —A0—e ce sehrel cuplevilen 220M pgstaceuple ("Magazine  ); 7 

cle COUNT esq) s-W bsdmdg —A =e -c)  oEhiCl tuple coum: PROM posuatuuple( magazine); * bsdmagdis 

sleep 2 

echo "Updating tuples.." 

psql -U bsdmag -c "UPDATE magazine SET title = 'UPDATED' || title; " bsdmagdb 

Pegi =-U bsdmag --pseu pager—-otr =c > (PAGE OUERY |)” bsdmagdb 

ee hee oe eh Ole 

SEZE ZS is sk Se rieNODE (awk {prank 55,7" 

SiZey GUPEnE Z— esgl UU bsdiad a -b ey othe) tiple Wen BROOM egstatruple | magazine); | bedmaqdis? 

SZ COUNiZ— psi bsciag kee SECT tuple weounE ROM ecstattuele (magazine) osdmagdls| 

sleep 2 

echo "Deleting tuples.." 

psql -U bsdmag -c "DELETE FROM magazine; “ bsdmagdb 

peq!=-U bsdmag —-psce pager—our -e ". | PAGE OUNRY)” bedmacgdb 

is ih Sr PLE NODE 

SIZE yo] ls lke PIPENODE |Pawle {print 25, 4 








echo “==========================" 
SIZE 1 T='expr $SIZE 1 / $SIZE 1! 
SIZE 1 P='expr $SIZE 1 / $PAGE SIZE' 
SlAn 2 expr Soiszne2 7) SolZk 
SIZE 2 P='expr $SIZE 2 / $PAGE SIZE' 
SIZE 3 T="expr $SI1ZE 3 / $SIZE 1! 
SIZE 3 P='expr $SIZE 3 / S$PAGE SIZE' 


echo "Filenode was SFILENODE" 
Schou oie Do bebe cua Gie ing. soo AE nO) 
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From the output readers can clearly see that the table 
started as empty, then we added data for 32 pages and 
after the update the relation doubled its data pages. 
That is because the inserted tuples were marked as 
expired from the veparz, and so a new copy of each tuple 
was inserted as new. Finally, after the pezzrz also the 
second copy of each tuple was marked as expired; this 
is the reason why the table storage retained its size (see 
Figure 3). The script of Listing 6 reports also a dump of 
the first and last page data as follows: 





Dump of the first data page 
io | Ip-tlags | xmim | xmax | t ctid 





Dump of the last data page 65 
lp | lp flags | xmin | xmax | t_ctid 
----}---------- +------ +------ +-------- 

1 | Li, 2026 | OZ | 


2 | 1 | 1026 | 2027 | (65,2) 
3 | L. | 1026. | 1027 | (65,3) 
4 | 1 | 1026 | 1027 | (65,4) 
5 | 1. | 1026 | L027 | (65,5) 





It is interesting to note that tuples in the first data 
pages are now marked as dead (flag = 3) while tuples 
in the last page (i.e., the last version of the deleted 
tuples) are marked as /iving (flag = 1); this means that 
tuples in the first page have not to be considered at all 
by running transaction, while tuples in the last page 
must be considered according to the snapshot visibility 
rules. 

Careful readers should have noted that the ouput of 
Listing 6 shows that data pages after an uvpparz have not 
exactly doubled, but are now a little more than the initial 
number (i.e., 66 versus 32 initial pages). The reason for 
this extra space is that the veparz changed the size of 
each tuple (changing the title column), as reported by the 
total size of live tuples. 


Vacuum 

As explained in previous sections, when a tuple is 
modified (either via vppare OF peLeTe) a new version of 
the tuple is stored. This fills data pages with o/d and no 





Listing 6b. A script to test MVCC and how data pages changes 


SehOe wee. RecGeiascmE. poate 
echo)” 
CenOu et 7e 84 beer pdare. Polat 
echo: ! 

Ze) 


oom oie mean ce. POLAN Sy colAn 3 RP apages | 





echo I ell 





echo "Dump of the first data page" 





LAST PAGE='expr SSIZE 3 P - 1' 
echo "Dump of the llast data page SLAST PAGE” 





[yest Zh er pages with oolZE LUPIE ED byes ton sol Ze COUND He ive Tuples |* 


[ 2 StZheZ er pages with SolZENUPIE 2 byes bor 2o172u (COUNT 7 ive cuples) 


(ene Olina. ol Ai OE Giese shia ine Wel eec ea) 


peqly-U bednag pseu pagcer-ort ce. 5 | PAGE OUERY).” becmagdb 


PACE OE Gey SMe Bn Ch ey rena esi gc ni hres eae sis ol ete aI a eee es We Gin rs sal ae 
Page Meems ( Geu tawe pagel Magazine ,~) Asi Fler) ) ORDER BY i> kIMii Ss; 
peq!-U Dedmag =—pser pager ori —o "5, PAG! QUERY MAST)” bscmagd 


(aLOUnd =) Zn 7a Sees eile) 
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more visible tuples. A special tool, called vacuum, can 
clean up no more visible tuples freeing storage space 
for new live data. Vacuum is a kind of swiss knife for 
PostgreSQL and can act over a single table or an entire 
database and has several aims. It is worth noting that 
vacuum can be invoked from a live connection or via 
the vacuumdb(1) (and its brother vacuumio(1)) Command 
line executable. There are several flavours of vacuum, 
mainly: 

standard: reclaim for free space but within data page 
boundaries (so will not free effective space until the last 
data page can be entirely erased); 


¢ full: is the most aggressive way of running it, and it 
will clean all the expired tuples in all the data pages, 
reorganizing living tuples; 

¢ analyze: used to update the internal statistics (used 
for instance by the query optimizer). Can be run even 
on a single column; 

¢ freeze: used to avoid xid wraparound (see later). 


Please note that the vacuum fu11 locks the entire table(s), 
and therefore is the most aggressive and less concurrent 
maintenance task. The script of Listing 7, if executed 
immediately after the one of Listing 6, provides the 
following output due to a vacuum full: 





Filenode was /postgresql/clusterl/base/16398/25307 
540672 
Size after VACUUM: 0 


Size before starting: 
[ 0 pages ] (around 0 times 


initial size) 
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As readers can see the table is now empty and the data 
storage is empty too. While running, the vacuum command 
reports: 


INFO: 
INFO: 


vacuuming ,public.magazine” 
pmagazine”: found 5000 removable, 0 nonremovable 


row versions in 66 pages 


which states that the expired 5000 tuples are going to be 
removed from the storage since they are no more visible 
to any running transaction. 

There is another important task that vacuum has to 
accomplish: avoid the xid wraparound. As explained in 
the previous sections, each transaction is identified by 
an unique progressive number, the xid, which is internally 
handled as a 32 bit integer. Sooner or later, the xid will 
wrap around and since tuples are visible to transactions 
with a lower xid than the running one, the database will 
be in a condition where younger transactions will have 
a xid that is lower (so in the past) that of older running 
transactions. To avoid this, PostgreSQL starts numbering 
transactions from a non-zero value (3) and vacuum freezes 
old committed transaction tuples with a xid equal to 
frozen-xid (2), so that such tuples will always be perceived 
in the past even after a xid wrap around. The effects of the 
freeze can be seen executing a vacuum Manually on the 
magazine table: 


bsdmagdb=# VACUUM FREEZE magazine; 
bsdmagdb=# SELECT xmin, cmin, cmax, xmax FROM magazine LIMIT 5; 
xmin | cmin | cmax | xmax 
------ $------}------+------ 
2 | 0 | 0 | 0 





after UPDATE 
all tuples are marked 
as expired and 
a new copy of each is stored 


after INSERT 








after DELETE 
all not-expired yet tuples 
are marked as expired 





Figure 3. Conceptual evolution of tuples within the magazine table while executing the example workload 
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To avoid accidental data loss, PostgreSQL starts 
complaining the needing for a vacuum when the next xid 
is coming near to 10 millions remaining values to the 
wrap around and deactivates itself when there is only 1 
million left. If this threshold seems to much high to you 
please remember that each SQL statement is executed 
in a transaction context, even when a transaction has not 
been explicitly started. 


Auto-Vacuum 

Having to remember to manually vacuuming a cluster 
can be hard, and therefore, starting from the 8 series, 
PostgreSQL embeds an auto-vacuum feature. If auto- 
vacuum Is enabled via the autovacuum = on parameter in 
the postgresq/.conf file, and you wait enough time before 
executing the vacuum of Listing 7, you will see that such 
manual vacuum is doing almost nothing since the table 
has been already vacuumed. Autovacuum launches a set 
of worker processes every specified amount of time; each 
worker vacuums a table if it is long since last vacuum 


for the table or if the number of expired tuples is greater 
than a computed threshold. It is possible to set per-table 
autovacuum as in the following: 


ALTER TABLE magazine SET (autovacuum_ enabled = false); 


Since vacuum could be a resource intensive operation, 
PostgreSQL provides a rich set of parameters for fine 
tuning of the auto-vacuum which are out the scope of 
this article. 


Micro-Vacuum and HOT 

Microvacuum is a page-boundary limited vacuum, which 
aim is to reclaim space within the same data page. 
Microvacuum is used in the HOT (Heap Only Tuple) 
subsystem: the idea is that if a tuple is modified only 
for out-of-index properties PostgreSQL should search to 
keep the new tuple version in the same data page, so to 
avoid an index update too. To do so, a microvacuum is 
done on the data page to free some space for the new 
version of the tuple, and then a pointers chain to the 
new tuple is placed to make the new version available to 
queries. Moreover, when a single data page is accessed 
(via SELECT, UPDATE OF DELETE) a space cleanup is performed 
to keep the data page as much clean as possible. 





Listing 7. Cleaning up expired tuples using vacuum 


41 / ban sh 


DBOID='oid2name -gq | grep bsdmagdb | awk '{print $1;}'' 
IINGIE) (SILI eesone (5 eC ae 


FILENODE=S { PGDATA} /base/$ {DBOID}/${TABLEOID] 
echo "Cleaning the magazine table..." 

ie vali SEL LENODE 

Ae be alle SHILENODE awk “(prank 5)! 


ls =Ih SPILENODE 
SUZE Zs Tees Obi ieNODEy |Vowk (printe55,) 7 





SCO) Y= =======222-—===------- == 
SIZE 1 T='expr $SIZE 1 / $SI1ZE 1! 
SIZE 1 P='expr $SIZE 1 / SPAGE SIZE! 
SIZE 2 T='expr $SIZE 2 / $SI1ZE 1! 
SIZE 2 P="expr SSIZE 2 / SPAGE SIZE! 
echo "Filenode was $FILENODE" 

ECNOn ze PelOre we uateI Ng spol Angi 


eclloe Size diver VACUUM. ~olAbec yl oo UE Zee pages.) 








TABLEOID='oid2name -U bsdmag -t magazine -d bsdmagdb -q | awk '{print $1;}'' 


psql -U bsdmag -c "VACUUM FULL VERBOSE magazine; " bsdmagdb 


(ecound SEA) Teimes simitra bee ize) 
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On The Web 

- PostgreSQL official Web Site: http://www.postgresql.org 
ITPUG official Web Site: http://www.itpug.org 
PostgreSQL 9.1 Data Page Layout: http://www.postgresql.org/ 
docs/9.1/static/storage-page-layout.html 
PostgreSQL 9.1 Documentation on Vacuum: /http:// 
www.postgresql.org/docs/9. 1/static/sql-vacuum.html 
Bruce Momjan, MVCC Unmasked (Talk at the Fourth Italian 
PGDay): momjian.us/main/writings/pgsql/mvcc.pdf 
Scripts and examples used in this article are available via 
GitHub repository at https://github.com/fluca1978/fluca-pg- 
utils 





In order to allow a better page-boundary vacuum, each 
table can have a specific fillfactor, that is percentage of 
free space to guarantee for updates. In particular the 
fillfactor specifies how much space can be consumed 
in a data page by tnszrr Commands, leaving the rest of 
the space free for urparzs. If a table is never updated the 
default fillfactor of 100 (full package) is the best, while if a 
table is often updated a lower fillfactor will preserve disk 
space in the long run. To see the effect of the fillfactor you 
can run again the script of Listing 6 setting a fillfactor of 40; 
the final result will be that the number of pages after the 
updates is the same as after the interts, since each page 
kept free space for new versions of the same tuples. 


Summary and Coming Next 
This article explained how PostgreSQL manages 
concurrency and how it stores tuples. Knowing how 
the internal storage works can be helpful in tuning and 
maintaning large database to perform at best. 

In the next article readers will see how to replicate 
a running PostgreSQL cluster into another running 
instance. 


LUCA FERRARI 
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Beowulf Clusters with 
DragonflyBSD 





There are two types of computing clusters: High availability 
(HA) clusters are designed so that if one computer fails, 

the other(s) take over its job. HPC clusters enable many 
computers to do the same job together so that processing 
power is increased. We're going to focus on the latter. 


What you will learn... 
¢ How to build a high performance computing (HPC) cluster with 
DragonflyBSD 


a Beowulf after the classic poem written sometime 

between 700 — 1000 AD. Beowulf technology is 
the result of a 1994 cooperative research project between 
NASA and several universities. Since DragonflyBSD 
development focuses so much on performance, it seems 
the best option fora BSD Beowulf. In fact, HPC clusters are 
one of the stated design goals of DragonflyBSD. 

There isn’t a software program called Beowulf. There 
are several solutions for implementing Beowulf. We'll 
use a common solution called MPICH2. Fortunately, 
DragonflyBSD offers a package for MPICH2. In a 
Beowulf, one computer is the master node. It controls all 
of the other nodes called clients. 

Let’s start with our master node, which I’ve named 
wolfmaster. | Know what you're thinking: Beowulf has an 
‘u’, and wolfmaster has an ‘o’. You’re inconsistent, Toby. | 
know. | just felt like doing it that way. Wolfmaster has an IP 
address of 192.168.0.10. First, we install MPICH2: 


A n HPC cluster on consumer grade hardware is called 


# okg Ladd moich2=1.3.1 
Next, use the adduser Command to add a user called 


wolf. Notice the UID number when you're done. Now 
that we have a /nome/woift directory we run a couple of 
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What you need 

¢ The ability to use the command line interface. 

¢« Two or more computers with DragonflyBSD 2.10.1 on the same 
subnet. Each computer must be running the same architecture 
(don’t mix 32-bit with 64-bit). 

« An understanding of what it means to compile a program. 


commands. Users from the other node will use this 
directory, so we give wolf's profile a umask allowing 
access for other users. 


# echo ‘umask 007’ >> /home/wolf/.profile 
Export /nome/wo1t aS an NFS share: 


# echo ‘/home/wolf -alldirs -network 192.168.0.0 -mask 
255.255.255.0" >> /etc/ exports 


To turn on NFS sharing at boot time, edit /etc/rc.cong, 
and add these lines: 


portmap enable="YES” 
nfs server enable="YES” 


mountd flags="—r" 


MPICH2 uses hostnames even if you tell it to use IP 
addresses, so if you don't have the names in a DNS 
server somewhere, you'll have to edit the hosts file like | 
did by adding the following lines: 


192.168.0.10 wolfmaster wolfmaster. 
192.168.0.11 wolfnode00 wolfnode00O. 
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| gave each node an alias of its name as well as its 
name followed by a “.” because MPICH2 wants to use 
the fully qualified domain name (FQDN). While installing 
DragonflyBSD, | did not provide an FQDN for the host 
name, so MPICH2 adds the *.” when looking for other 
nodes. 

In this article, we're building a cluster with only two 
nodes. Beowulf supports up to 1024 nodes. You wouldn't 
want to update 1024 hosts files each with 1024 entries. 


These next steps will become clear later on in the article: 


# mv /etc/hosts /home/wolf/hosts 
# ln -s /home/wolf/hosts /etc/hosts 


That’s all we need to do as root on wolfmaster. Now log 
in as the wolf user. We have a bit more work to do for 
password-free SSH. 


Begin Adding Node 
$ ssh-keygen -b 2048 -f ~/.ssh/id rsa -t rsa -N ,” 


Note that after the -n argument, there are two double 
quotes with nothing between them, not even a space. 
This ensures that no passphrase will be required when 
doing remote login. Then copy the file ia rsa.pub into 
another file authorized keys: 


> CC *7. 85h 

S cp id _rsa.pub authorized keys 

$ chmod 644 ~/.ssh/authorized keys 

S chmod 755 ~/.ssh 

We could start NFS now by starting/restarting nfsd and 
mountd, but we want to be sure that NFS comes up at 
boot time. Let’s restart wofmaster now. Time to move 
onto the client node I’ve named the client wolfnodeOO, 
and it has an IP address of 192.168.0.11. 

Start by invoking the aaduser Command again. Name the 
user wolf. When prompted for the UID, type in the same 
number as the UID of the wolf user on the master node. 
Use pxg_rada to install MPICH2. 


echo ‘wolfmaster:/home/wolf /home/wolf nfs ro 0 0’ >> 


jetc/tfstab 


Remember when we moved the /etc/hosts file on 
wolfmaster to /nome/wolf/hosts, and then created a symbolic 
link for /etc/nosts? IN a moment, wolfnodeO0 is going mount 
wolfmaster’s /nome/wolf as it’s own. Creating a link from an 
NFS mount won't work; the following magic will prevent us 
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from having to edit the /etc/nosts file on any client nodes. 
The /etc/nosts file will be updated at the top of every hour: 


# echo ‘0 * * * * root cp -f /home/wolf/hosts /etc/hosts’ 


>> feto/crontab 


We could mount the NFS export with the mount 
command, but again: we want to make sure that it does 
its thing at boot time. Restart wolfnodeO0O. By running the 
af Command, you should see that the last line reads: 


wolfmaster:/home/wolf [some information about blocks] 


/home/wolf 


To add more nodes, first modify the /nome/wolf/hosts 
file. Then start from the place in this article that says 
BEGIN ADDING wnopE. Substitute wolfnodeO7 ...02, 03, etc 
for wolfnode00. When you get to this point, you'll have 
successfully added another node. 


End Adding Node 
Back to the master node. Log on as wolf. Execute this 
command: 


S ssh wolfnode00 hostname 


You should not have to enter a password, and you 
should find that the hostname (wolfnodeOO) of the client 
(not the master) is returned. The last thing to do before 
we can start testing MPICHZ2 is to create a file ON /nome/ 
wolf. | called it nodes. The contents should be the name 
of each node with one node per line, like this: 


wolfmaster 


wolfnode00 


Beowulf programs are executed with the mpiexec 
command. There are two switches that we need for 
basic usage: 


¢ -f specifies the file name with the list of nodes. In my 
CaSe@, /home/wolf/nodes 

¢ -n specifies the number of nodes to run a program on. 
If you specify a number that is greater than the number 
of nodes you have, then performance will decrease. 

To test the setup, try this: 


Smpiexec -f /home/wolf/nodes -n 2 hostname 


The result should be: 
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Table 1. MPICH2 compilers 











mpicxx C++ 
wolfmaster 
wolfnode00 


Notice that the hostname Command ran independently 
on each node. That’s why we have two results instead 
of one. For a program to run with the performance of 
our HPC cluster, it must be compiled with the MPICH2 


libraries. MPICH2 includes utilities for doing this: 
Table 1. 
Fortunately, there are example programs to try. 


Unfortunately, they don’t come with DragonflyBSD’s 
MPICH2 package. Let's download the source tarball for 
MPICH2: 


S Curl =o moich2=1.5.lyvtar.gz + 
http://www.mcs.anl.gov/research/projects/mpich2/downloads 
/tarballs/ ls 3eljmpich2-1. 361 sear.qz 

Star xvia moichy-1,3.1.tar.¢gz 

> €C- mpichn2—1 341 


The default install of Dragonfly¥BSD doesn’t include 
Fortran support: 


S ./configure --disable-f77 --disable-fc 


S make 
First, let’s try one of the precompiled examples: 


S$ mpiexec -f /home/wolf/nodes -n 2 /home/wolf 


/mpich2-1.3.1/examples/cpi 


This should return pi. In order to really test that our 
cluster is faster than one computer, we'll need to compile 
one of the examples. The MPICHZ2 libraries are in 
mpich2-1.3.1/lib, so the command looks like this: 


S$ mpicc -o /home/wolf/icpi -L /home/wolf/mpich2-1.3.1 
/lib /home/wolf/mpich2-1.3.1/examples/icpi.c 


The icpi program will ask you to input how many times to 
run the pi algorithm. | am using two 2 GHz Core 2 Duo 
computers, and 10,000,000,000 (ten billion) turned out 
to be a good test. First | ran it without mpiexec. 


S$ /home/wolf/icpi 
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Result: 23.6 seconds. Let’s now run it with mpicc, but on 
only one node: 


S$ mpiexec -f /home/wolf/nodes -n 1 /home/wolf/icpi 


Result: 23.6 seconds. Now the moment you've been 
waiting for. Let’s run it on both nodes: 


S$ mpiexec -f /home/wolf/nodes -n 2 /home/wolf/icpi 


Result: 11.8 seconds... Success! 

When building your Beowulf, the speed of your network 
is paramount. Consider my two-node Beowulf. Calculating 
pi doesn’t use much memory. Everything happens in the 
caches of the CPU’s. What about more memory intensive 
programs? My nodes each have an 800MHz front side 
bus (FSB). I’m running a 32 bit OS. Multiply that by my 32 
registers, and you get 25.6 Gbps as opposed to the data 
transfer rate of my network: 1Gbps. More likely a Beowulf 
today would consist of computers with a 1.6GHz FSB and 
a 64 bit OS. Now our RAM is communicating with the CPU 
at 102.4 Gbps. I’m not even going to get into the effects of 
dual-, triple-, or quad-channel memory (mostly because the 
known benchmarks are inconclusive) http://en. wikipedia.org/ 
wiki/Multi-channel_memory_architecture#Performance. 

Clearly I’m better off using a computer with two CPU’s 
instead of a two-node Beowulf where each node has a single 
CPU. Beowulf is useful because it’s far less expensive to 
build a cluster with 20 computers instead of buying a single 
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For more information on Beowulf and 


a check out the following web sites: 
Beowulf site (including the history of Beowulf): http:// 
www.beowulf.org 
MPICH2_ site: hAttp://www.mcs.anl.gov/research/projects/ 
mpich2/ 

DragonflyBSD site: http://www.dragonflybsd.org 
Information about loss of a node: http://mpich-v.Iri.fr/ 
papers/MPICH-V2. pdf 

McGill Pulsar Group: http://www.physics.mcgill.ca/~pulsar/ 
Welcome.html 








computer with 20 CPU's; still, you can see why you wouldn't 
want to run a Beowulf on a slow network connection. 

When is a Beowulf useful? When your program requires 
lots of processing power, but little disk access. Aweb server 
or database server would not be a good use of Beowolf. 
Those sorts of programs are more disk and memory 
intensive relative to the processing power that they need. 
Projects that require lots and lots of math computations 
are the best use for Beowulf. For this reason, you'll find 
Beowulfs in the science departments of many universities. 

Let's wrap things up by talking about a real world 
example of why someone might want a Beowulf. A friend 
of mine named Brian has a degree in engineering, and he 
builds robots for fun. Typically, he’d build them for Battle 
Bots (www.battlebots.com). In 2008, he and his team 
decided to enter a competition sponsored by NASA to 
build a robot for moving dirt around on the Moon. 

Robot building is an expensive hobby. Brian is always 
complaining about the price of titanium. Rather than build 
several prototypes, they had to get the design right the 
first time. They needed to run computer simulations to 
predict how the final product would perform. As amateurs, 
there’s no way that he and his team could afford time on 
a supercomputer. 

Fortunately for Brian, he has friends in the e-waste 
industry. Many organizations dispose of computers that 
still function. Brian was able to obtain about fifty working 
2 GHz Core 2 Duo machines for free. With Beowulf, they 
were able to run the necessary simulations for building 
their robot at 2 GHz * 50 nodes = 100 GHz. Of eight 
competitors, Brian’s team was the only robot that could 
complete the objectives of the competition. 


TOBY RICHARDS 
Toby Richards has been a network administrator since 1997. 
Each article comes straight from the notes that he takes when 
doing a new project with *BSD. Toby recommends bsdvm.com 
for your hosting needs because they provide console access to 
your virtual machine. 
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TIPS&TRICKS 





Npppd: Easy PPTP VPN 





with OpenBSD 


Have you ever needed to set up a VPN for Microsoft 
Windows or Mac OS X users? From this article you will find 
out how to configure OpenBSD and npppd to provide PPTP 


and L2TP VPN's in a few easy steps. 


What you will learn... 
¢« How to setup a PPTP/L2TP VPN server 


source tree and this software can act as a PPTP/L2TP 
VPN server and also as a PPPOE server. 

Because npppd is still under active development and 
still missing some features, it is not linked to the standard 
build yet, so to install the program you first need to build it 
from OpenBSD source tree. 

First install OpenBSD-current or wait for 5.1 which will 
be released around May 1st, 2012 (Listing 1). 

Now you will have two new programs installed under 
/ust/sbin: npppd, the PPTP/L2TP daemon and npppctl, the 
userland utility. 

First you want to configure the software: Enable the gre, 
esp and pipex protocols using sysctl: Listing 2. 

Edit /etc/sysctl.conf accordingly to make these changes 
persistent across reboots. 

Create a directory under /etc where you will put your 
configuration files: 


n January 2010, npppd was imported into the OpenBSD 





What's Wrong With Poptop? 


Poptop has many features, runs on many platforms and 
can be installed with one simple command on OpenBSD: 


pkg add poptop. 

However, Poptop is not designed with security in mind (it 
has no privilege separation), it does not provide RADIUS au- 
thentication (without unofficial patches to PPP),and, at least 
on OpenBSD, it does not perform very well. 
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What you should know... 
« Basic OpenBSD tasks 
¢ Basic TCP/IP and routing knowledge 


S$ sudo mkdir -m 0755 /etc/npppd 
Create the file: 
Listing 3. 

With this example configuration, the tunO interface is 
used to concentrate VPN access and the 192.168.255.0/ 
25 network is assigned to users. 


/etc/npppd/npppd.conf configuration 


Listing 1. /nstall npppd 


= cd /use 

© Cvs —-G@ amoncvedanoncvs. fr openbsd.org:/evs checkout —P 
Ssec/Ust.sbin, 4Maketilelinc, mpopct!, noppd} 

Gd) src/use-sbin/moppd 

make 

sudo make install 

el ony mosieeel! 


make 


ey i 


sudo make install 


Listing 2. Enable needed protocols 


# (for PPTP) 
# (for IPSEC) 
# (for PIPEX) 


S sudo sysctl net.inet.gre.allow=1 
S sudo sysctl net.inet.esp.allow=1 


S$ sudo sysctl net.pipex.enable=1 
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Listing 3. Npppd configuration 


[npppd- cons | 
Ibert hace Fisk: tun 
interface.tun0.ip4addr: LO Gein, Joe L 


# IP address pool 
pool.dyna pool: OZ W682 55.0725 
pool spool: O02 sie s=255.07 24 


+ Locale tle sauEhenmk leat mom 


aul s Locale pean lista: local 
allehe locallecealmeacerli ctr: /etc/npppd/npppd-users.csv 
realm.local.concentrate: tund 


# RADIUS authentication / accounting 


#auth.radius.realm list: radius 
#auth.radius.realm.server.address: V2 7 2050. e312 
#auth.radius.realm.server.secret: password 


foUEM fags teal ccUNoot yc, c0droos. I 700 (rel. 








(AUN rog@tise ted lille sek yCieccehe reac e won 


#realm.radius.concentrate: tund 
leoemuu: 1400 

auth.method: mschapv2 chap 

LoCo. dns POR Imamy: POA Looe lO 254 
#ipcp.dns secondary: 2 6s 6. 20 
LOCO MON Sseprl Mary: LOZ ose hoy 3254 
#lpCp-mbme secondary. DOA. hoes 6.2 0 
#ipcp.assign fixed: true 
PjipCpecdss lOMeUsetoc lec. true 
pptpd.enabled: true 

pptpd.ip4 allow: O20 02070 


# L2TP daemon 


12tpd.enabled: true 
12tpd.ip4 allow: 020. 0.070 
UZEpdeseqitne yi psec: false 


Listing 4. Setup vpn users 


Username, Password, Framed-IP-Address, Framed-IP-Netmask, Description, Calling-Id 
userl, passwordl 


user2,password2 
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TIPS&TRICKS 


It is better to choose a network setup for your VPN that 
is different from the one you are using for your internal 
interface, otherwise your VPN users could have problems 
reaching your LAN. 

When npppd starts, it will setup a tun interface with the 
IP address 192.168.255.1 and will authenticate users by 
reading the password file /etc/npppd/npppd-users.csv. 

npppa Could also authenticate users using a RADIUS 
server. 

Once you have configured your LAN DNS (ipcp.dns_ 
primary) and netbios over TCP/IP (ipcp.nbns_ primary) 


Listing 5. pptp vpn client setup 
vpn pptp: 


set authname userl 

set authkey passwordl 
set mppe 128 stateless 
disable protocomp 
deny protocomp 


disable ipv6ocp 


Listing 6. Monitoring vpn usage 


# npppctl session brief 


Tunnel From 


Start Time 2 20N2/ 02/07 MAs05 241 
Elapsed Time P0967 sec 

Input Bytes 2 OZ S.0 SIOZ (sy 10) Wills: 
Input Packets > 38364 

ao UierE ene me sl (0.05) 

Output Bytes 2 oO Zo) MY 
Output Packets : 46374 

CUEOW ek rorag Zo sO.) 





set device "!/usr/local/sbin/pptp --nolaunchpppd A.B. 


Proto Tunnel From 


PPEP Ost 22/7—1 9-dyieol 3 5-G.proviger. ies o2 30 


Roomba Assigned IPv4 Username 
OO? CGO oO! renova nal 

# npppctl session all 

Ppp Id = 8 
Poole ee 
Username * ‘Glovanni 
Realm Name 7 local 
Concentrated Interface : tun0 
Assigned IPv4 Address Pleo Oe eo Oe 
Funmeils Prokoce:l: pee Ene 


SNOsuZ22 9 -dyn. loo. Provider. le 


i hour and 56 manutes) 


servers, you can start adding users to the password 
file. 

The password file /etc/npppd/npppd-users.csv isa CSV file 
with minimal information about users: Listing 4. 

Only the user and password fields are required and 
the second and third fields are used to assign the same 
IP address every time a specific user connects to the 
VPN. 

The password file should be kept secured with standard 
file access permissions, if you want more security radius 
should be used instead. 


Goby 
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Listing 7. Routing configuration in a nat setup 


inet LOZ alGe 2255 oi 
eoue wacd milo og a2 520) 2 oat O72 og e 255i 


Listing 8. /etc/ipsec.conf configuration to let I2tp protocol work 


ike passive esp transport \ 
PLOtOuIdperromany Ce any poet. 170i 
Main auth “hmac-sha" enc "3des" group modoz04e \ 
quick auth “hmac-—sha" enc “aes” \ 


psk “password” 











Your PPTP setup is now complete and you can start 
npppd with -d debug mode enabled or the -D daemon 
mode parameter. You can now try your VPN: 


# /usr/sbin/npppd -D 


You can connect to your VPN with Microsoft Windows, 
Mac OS X, *BSD or Linux. 

For example, you can use pptp from ports to connect to 
your VPN. 

To install pptp from ports: 


S sudo pkg add pptp 


edit /etc/ppp/ppp.conf (Listing 5). 

Modify ,A.B.C.D” to your VPN public IP address and 
user and password according to your setup. 

Now you can connect to your VPN: 


S sudo ppp -ddial vpn pptp 


You can monitor connected users from your VPN server 
console using npppcti: Listing 6. 

lf necessary, you can even disconnect VPN users from 
the VPN server with the command npppctl clear all. 


Tips & Tricks 

lf your VPN server is under NAT, you should setup a route 
to connect your LAN and your VPN networks. 

# route add 192.168.255.0/25 192.168.255.1 

To make these changes permanent, you should edit /etc/ 


hostname.tuno and add to the file: Listing c. 
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L2TP VPN Setup 
Layer 2 Tunneling Protocol (L2TP) is a tunneling protocol 
used to support virtual private networks (VPNs). It does 
not provide any encryption or confidentiality by itself; it 
relies on an encryption protocol that it passes within the 
tunnel to provide privacy. 

In OpenBSD /etc/ipsec.conf is used to setup encryption 
parameters (Listing 8). 

IPsec is now configured and you can run the IPsec 
daemon: 


# isakmpd -Kv 


Execute ipsecctl to notify isakmpd of configuration 
changes. 


# ipsecctl -f /etc/ipsec.conf 





Caveats 

npppd is still a work in progress and some features are still not 
implemented: 

there is no proxyarp support so you cannot use the same 
network space for both LAN and VPN. 

There is also no support for NAT-T IPsec so if your VPN server is 
under NAT, it will work only with the PPTP protocol. 








GIOVANNI BECHIS 

Giovanni Bechis lives in Italy with his wife and son.He is an 
OpenBSD developer and the owner of SnB, a software house 
which provides web design and hosting solutions based mainly 
on *BSD systems. 

He can be found at http://www.snb. it. 

For many years, Poptop has been the only software available to 
create PPTP VPN connections from Linux/BSD. 

Poptop has some issues that lead a Japanese software house 
start writing their own PPTP/L2TP VPN server. 
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Anatomy 





of a FreeBSD compromise (Part 4) 


Continuing our security series, we will look at the 


vulnerabilities on our test network 


What you will learn... 
« How to use Nessus, exploitation tools and payloads 


a system we continually needed to move from the 

general to the specific, and to identify the most 
vulnerable system on our network depending on what 
services were running on it (Figure 1). We are now going 
to attempt to run a successful exploit on the machine with 
the most ports open, and to improve our chances, we will 
use a legacy version of FreeBSD 6.1 with Apache 1.3.37 
rather than the current release. 


Z rom the last article, we discovered that to penetrate 


Aim of the attack 
From the hackers perspective, the aim is to release either 
a Zero Day Vulnerability (2) or a well documented proven 


General Recuerc 





Specific 














Figure 1. Attack strategy 
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What you should know... 


¢ BSD and network administration skills 


attack (4) against an unprotected target host. A well 
protected or patched machine will not be vulnerable either 
because there is no known attack (1) or the machine has 
been patched against exploits previously found in the wild 
(3) — see (Figure 2). An undiscovered attack is low risk as 
the hackers are not aware of it, and at the same time as 
code is reviewed and tested, developers will (hopefully) 
preempt obvious holes. The highest risk is where a known 
attack is in the wild, yet the system is not patched or 
modified to counter this (4). The security footprint of the 
system is crucial — it can be argued that there are many 
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Anatomy of a FreeBSD compromise (Part 4) 


Table 1. Potential Targets 


Potential Targets 


Hacker 192.168.0.131 NTOP, TCPDUMP, NMAP 


192.168.0.141 TCPDUMP 


> Border 192.168.0.254 NTOP, TCPDUMP, NMAP 








Table 2. Open Ports on victim 


Open ports on 192.168.0.254 


25/tcp open smtp 
53/tcp open domain 
80/tcp open http 
110/tcp open pop3 
139/tcp open netbios-ssn 
143/tcp open imap 
587/tcp open submission 
631/tcp open ipp 
3128/tcp open squid-http 





more exploits for certain platforms because they are 
more popular, so it will be interesting to see if the hackers 
develop more Linux / *BSD attacks with the growth in 
popularity of mobile devices. 


Methodology 

The potential types of attack include Cross-Site Scripting 
(XSS), Brute Force Password, Buffer Overflows, SQL 
Injection, Denial of Service etc. There are many ways of 
delivering a payload via shellcode, including C / Bash / 


Table 3. Further reading and resources 





Failure 





Payload 


Success 











Figure 3. Key toa successful exploit 


Perl, custom scripts, executables etc. as well as using 
tools such as Metasploit. There is no magic bullet — a 
single tool or program that will guarantee a successful 
attack under all circumstances as there are too many 
variables to consider. A script-kiddie may pick up a file 
that gains access to some machines, but this may be 
ineffective across versions, patches, architectures or 
potentially even language versions or the amount of 
memory installed on the machine. In other words, off 
the shelf attacks are just the starting point for the serious 
hacker, as each exploit has to be carefully tuned to the 
victims’ environment (Figure 3). The vulnerability belongs 
to the system, the exploit and payload to the attacker. 
While the exploit opens the door to the attacker, the 
payload does the actual damage. As the number of 
available exploits and payloads increase, it becomes very 
time consuming to quickly find the best vulnerability. This 


Further reading 





URL 


Metasploit tutorial (Functionality may differ slightly from BSD) 


http://www.offensive-security.com/metasploit- 
unleashed/Main_Page 


Obtaining a Nessus activation code 


http://www.tenable.com/products/nessus/nessus- 
plugins/obtain-an-activation-code 


Dictionary of publicly known information security vulnerabilities http://cve.mitre.org 


Global cooperative cyber threat / internet security monitor and alert system http://isc.sans.org 


World class information security training and penetration testing http://www.offensive-security.com 


Carnegie Mellon University's Software Engineering Institute http://www.cert.org 
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Table 4. mfsconsole useful commands 


Search for exploits or modules Search apache 


e.g. apache 

Show information for exploit info 
Show all exploits show exploits 
Show all payloads show payloads 


Show options for current 
exploit or module 


show options 


set PAYLOAD linux/x86/exec 
help 


Load payload linux/x86/exec 


Show help 


is where Nessus and Metasploit make such formidable 
tools in the security professionals’ armory, as they open 
the door to automated vulnerability discovery and attack. 
The latest version of Backtrack (5R1) includes Armitage, 
a Java GUI driven attack management tool which greatly 
aids in discovering and executing vulnerabilities depending 
on the O/S type and the network operating environment. 
It will attempt to find the best match of exploit to victim. 
Metasploit also provides the ab autopwn utility which offers 
similar functionality. 


With the availability of reliable desktop virtualization, 
Ready Baked versions of security toolkits such as 
Backtrack are available as Vmware images and ISOs. 
One other benefit of releases such as Backtrack is 
the wide inclusion of other tools not available on the 
“BSD platform such as Maltego, which allows the 
forensic investigator to build an accurate picture of the 
environment and data mine the results. At the time of 





Figure 4. Nessus under Backtrack 
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Figure 5. Logging in to Nessus 


writing, Backtrack R5 also supports the latest release 
of Nessus, whereas Backtrack R5r1 does not. Tenable 
Security document on their blog how to run Nessus 
under Backtrack, but this functionality is missing from 
Backtrack R5r1 for some reason (Table 3). 

Installing tools such as Metasploit, Nessus etc. under 
“BSD is not quick — while packages are available, it 
is best to use ports. Although this is not difficult for 
experienced *BSD users, it is time-consuming to compile 
all the dependencies from scratch. For this tutorial | 
will be using a combination of Backtrack 5 (+Nessus), 
Backtrack 5r1 (+Armitage), and FreeBSD 9.0 running 
under Virtualbox. 

To install Metasploit under FreeBSD 9.0 (as root): 


pkg_ add -r ruby 

pkg add =r rubylé-iconv 

cd /usr/ports/security/metasploit 
make install clean 

cd /usr/local/share/metasploit 
svn upgrade 

msfconsole 


svn up 








Figure 6. Adding ascan for 192.168.0.254 
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Figure 7. Overview of vulnerabilities for 192.168.0.254 


This took > 50 minutes on my VM. Type exit to leave 
Metasploit. If you require DB support, this will need to be 
added separately either using pkg _ aaa or compiling from 
source. 


Step 1 — Identify exploits 
The quality of up to date information is critical when 
planning an attack. There are many useful security sites 
on the World Wide Web, but one of the the best tools 
to automate vulnerability discovery and assessment 
is Nessus. While a feed (which contains all the latest 
vulnerabilities) for home use is provided for free, if you 
wish to use Nessus in a commercial environment you 
must purchase a professional feed to meet the terms of 
their license. The other alternative is to manually research 
vulnerabilities, or to use the tools available with Metasploit 
and Armitage. There are other tools available with widely- 
ranging abilities, so Your Mileage May Vary (Table 3). 

lf you want to use Nessus, either boot from the Backtrack 
ISO or virtual image. You will be asked for a login name 
and password which is root and toor respectively. Follow 
the instructions to startx and you will be presented with 
the standard Gnome interface. 
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Figure 8. HTTP vulnerabilities 
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Figure 9. Even the printer is not immune 

Click on Nessus Register to register for a home feed via 
your browser (Figure 4). An activation code will be sent to 
you via email. When you receive this, open a terminal and 
type the following to register: 


/opt/nessus/bin/nessus-fetch -register @xxxx-xxxx-xXXXxX-— 


XXXX-XXXX@ 


Where xxxx-xxxx-xxxx-xxxx-xxxx IS your authorization key. 
Now add a user with admin rights: 


/opt/nessus/sbin/nessus-adduser 


You may update your nessus plugins with the following 
command: 


/opt/nessus/sbin/nessus-update-plugins 


Start Nessus from the menu using Nessus Start (Figure 
4). You can now login to Nessus either using the Firefox 
browser supplied in Backtrack, or via another machine. 
Point your browser at https://localhost:8834 and login 
with the credentials you supplied earlier (Figure 5). There 
may be a delay while Nessus loads all the plugins. Add 
the target host you want to scan under internal network 
scan and Launch scan (Figure 6). After a while you will 
be presented with a set of reports that you can drill down 
through. From our test, we can see that the web-server 
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Figure 10. Running Armitage 
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Figure 11. Armitage GUI with hosts added and scanned 





is the most vulnerable with 23 vulnerabilities, 9 serious 
(Figure 7 — 9). 


Metasploit runs pretty much identically under FreeBSD as 
it does under Backtrack, so we will use the BSD version 
to perform a test exploit for open telnet ports. Ensure your 
exploits etc. are up to date, and execute the following, 
replacing 192.168.0.0 with your network: 


msfconsole 


use auxiliary/scanner/telnet/telnet encrypt overflow 
set RHOSTS 192.168.0.0/24 
set THREADS 50 


run 


lf you have a vulnerable device on your network, it will be 
shown as vulnerable. 
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Figure 12. Finding attacks 
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Figure 13. checking for http exploits 

Tips 

Useful msfconsole commands are listed in Table 4. A 
good exploit to try on *BSD Is exploit/freebsd/telnet/ 
telnet encrypt keyid aS It Is new and has an excellent 
chance of success. As the framework is script based, 
each script will require different parameters, info and help 
are your friends. 


Using the Backtrack 5r1 ISO, login as above and run 
Armitage (Figure 10). Login to the server with the 
provided username and password, and run RPC support 
when requested. After a short delay, you will be presented 
with the Armitage GUI (Figure 11). Add the hosts you want 
to test via the hosts menu, and right click the PC to scan 
and update the O/S type if known. Find attacks for these 
devices on the Attack menu, and when right clicked you 
can run all the exploits (e.g. http) in one go by running 
check exploits... against the host (Figure 12-13). If an 
attack is successful, it will change color and this will be 
shown in the console at the bottom. To hunt for exploits 
across all devices, use the Hail Mary option. 
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What has your server vendor done for 
BSD lately? Probably, not much. 





Work with a vendor that supports the 
operating system you love! 


iX is the corporate sponsor of the PC-BSD® Project, a major corporate donor to the FreeBSD Foundation, 
and leads the FreeNAS™ development team -- all while employing some of the most brilliant minds in 


the FreeBSD® community. For BSD hardware and software expertise, look no further. 


1-855-GREP-4-IX 
http://www.iXsystems.com/community 








